Transparent user interface integration between local and remote computing environments

ABSTRACT

Methods and systems for transparent user interface integration between remote (“published”) applications and their local counterparts are described, providing a seamless, unified user experience, and allowing integration of a start menu, dock, taskbar, desktop shortcuts, windows, window and application switching, system tray elements, client-to-host and host-to-client file type association, URL redirection, browser cookie redirection, token redirection, status message interception and redirection, and other elements. These methods and systems further enhance theme-integration between a client and remote desktop or virtual machine by remoting all UI elements to a recipient for generation, including text controls, buttons, progress bars, radio buttons, list boxes, or other elements; presenting them with the receiver&#39;s product and OS-specific UI; and returning status back to the sender. This may achieve a more unified and transparent UI integration. Furthermore, international text may be correctly received in cross-language environments, or translated into the language of the presenting environment.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of and claims priority to U.S. patentapplication Ser. No. 13/410,659, filed Mar. 2, 2012, and entitled“TRANSPARENT USER INTERFACE INTEGRATION BETWEEN LOCAL AND REMOTECOMPUTING ENVIRONMENTS,” which claims priority to U.S. ProvisionalPatent Application Ser. No. 61/448,716, filed Mar. 3, 2011, and entitled“SYSTEMS AND METHODS FOR TRANSPARENT USER INTERFACE INTEGRATION BETWEENLOCAL AND REMOTE COMPUTING ENVIRONMENTS,” the disclosures of each ofwhich are incorporated by reference herein in their entirety and madepart hereof.

FIELD

The present disclosure relates to methods and systems for transparentuser interface integration between local and remote computingenvironments. In particular, the present disclosure relates to methodsand systems for providing a unified desktop experience of locallyexecuted applications and remotely executed applications withlocally-presented graphics.

BACKGROUND

In some environments for integrating a display of remotely generated orvirtual desktop environment on a remote computing device with locallygenerated desktop environments on a local computing device, applicationsmay be executed either on the remote computing device or the localclient computing device, to take advantage of the processor and memoryof the client. This may be done, for example, for multimedia purposes,device access issues, localization requirements, assisted computingdevices, etc. However, these applications are presently difficult orconfusing to use.

BRIEF SUMMARY

The following presents a simplified summary of the invention in order toprovide a basic understanding of some aspects of the invention. Thissummary is not an extensive overview of the invention. It is notintended to identify key or critical elements of the invention or todelineate the scope of the invention. The following summary merelypresents some concepts of the invention in a simplified form as aprelude to the more detailed description provided below.

In one embodiment, the methods and systems described herein provideintegration between remote (“published”) applications and their localcounterparts. In another embodiment, this functionality provides aseamless, unified user experience. In still another embodiment, thisfunctionality allows integration of a start menu, dock, taskbar, desktopshortcuts, windows, window and application switching, system trayelements, client-to-host and host-to-client file type association, URLredirection, browser cookie redirection, token redirection, statusmessage interception and redirection, and other elements.

In some embodiments, the methods and systems described herein enhancetheme-integration between a client and remote desktop or virtual machineby remoting all UI elements to a recipient for generation, such as textcontrols, buttons, progress bars, radio buttons, list boxes, or otherelements; then presenting them with the receiver's product andOS-specific UI; and returning status back to the sender. This mayachieve a more unified and transparent UI integration. Furthermore, insome embodiments, international text may be correctly received incross-language environments, or translated into the language of thepresenting environment.

The details of various embodiments of the invention are set forth in theaccompanying drawings and the description below.

BRIEF DESCRIPTION OF THE DRAWINGS

A more complete understanding of the present invention and theadvantages thereof may be acquired by referring to the followingdescription in consideration of the accompanying drawings, in which likereference numbers generally indicate identical, functionally similar,and/or structurally similar elements, and wherein:

FIG. 1A is a block diagram depicting an embodiment of a networkenvironment comprising local machines in communication with remotemachines;

FIGS. 1B-1E are block diagrams depicting embodiments of computers usefulin connection with the methods and systems described herein;

FIG. 2 is a block diagram depicting one embodiment of a system fordisplaying on a local machine graphical data generated on the localmachine and graphical data generated on a remote machine;

FIG. 3A is a block diagram depicting another embodiment of a system fordisplaying on a local machine graphical data generated on the localmachine and graphical data generated on a remote machine;

FIG. 3B is a flow diagram of an embodiment of a method for enumeratingpublished applications and redirecting application initiation requeststo a local machine;

FIG. 3C is a flow diagram of an embodiment of a method for displayingremote status messages in a local format;

FIG. 4A is a block diagram depicting one embodiment of integration oflocal and remote application windows in a full-screen remote desktop;and

FIG. 4B is a block diagram depicting another embodiment of integrationof local and remote application windows in a local desktop with awindowed remote desktop.

FIG. 5 illustrates a legacy logon status indicator dialog(host-generated and host-rendered).

FIG. 6 illustrates a logon status indicator dialog (host-generated butclient-rendered) according to an illustrative embodiment.

DETAILED DESCRIPTION

In the following description of the various embodiments, reference ismade to the accompanying drawings, which form a part hereof, and inwhich is shown by way of illustration various embodiments in which theinvention may be practiced. It is to be understood that otherembodiments may be utilized and structural and functional modificationsmay be made without departing from the scope of the present invention.The invention is capable of other embodiments and of being practiced orbeing carried out in various ways. Also, it is to be understood that thephraseology and terminology used herein are for the purpose ofdescription and should not be regarded as limiting. Rather, the phrasesand terms used herein are to be given their broadest interpretation andmeaning. The use of “including” and “comprising” and variations thereofis meant to encompass the items listed thereafter and equivalentsthereof as well as additional items and equivalents thereof. The use ofthe terms “mounted,” “connected,” “coupled,” “positioned,” “engaged” andsimilar terms, is meant to include both direct and indirect mounting,connecting, coupling, positioning and engaging.

One or more aspects of the invention may be embodied in computer-usableor readable data and/or computer-executable instructions, such as in oneor more program modules, executed by one or more computers or otherdevices as described herein. Generally, program modules includeroutines, programs, objects, components, data structures, etc. thatperform particular tasks or implement particular abstract data typeswhen executed by a processor in a computer or other device. The modulesmay be written in a source code programming language that issubsequently compiled for execution, or may be written in a scriptinglanguage such as (but not limited to) HTML or XML. The computerexecutable instructions may be stored on a computer readable medium suchas a hard disk, optical disk, removable storage media, solid statememory, RAM, etc. As will be appreciated by one of skill in the art, thefunctionality of the program modules may be combined or distributed asdesired in various embodiments. In addition, the functionality may beembodied in whole or in part in firmware or hardware equivalents such asintegrated circuits, field programmable gate arrays (FPGA), and thelike. Particular data structures may be used to more effectivelyimplement one or more aspects of the invention, and such data structuresare contemplated within the scope of computer executable instructionsand computer-usable data described herein.

For purposes of reading the description of the various embodimentsbelow, the following descriptions of the sections of the specificationand their respective contents may be helpful:

Section A describes a network environment and computing environmentwhich may be useful for practicing embodiments described herein; and

Section B describes illustrative embodiments of systems and methods forintegrating local and remote applications and desktops to provide aseamless user experience.

Section C describes a specific illustrative embodiment for seamlesswindows using virtual channels.

Section D describes illustrative embodiments using reverse seamlessfunctionality.

Section E describes interface configuration details according to anillustrative embodiment.

Section F describes a protocol for a control channel virtual channelaccording to an illustrative embodiment.

Section G describes a Transparent User Interface Integration virtualchannel according to an illustrative embodiment.

Section A. Network and Computing Environment

Referring now to FIG. 1A, an embodiment of a network environment isdepicted. In brief overview, the network environment comprises one ormore local machines 102 a-102 n (also generally referred to as localmachine(s) 102, client(s) 102, client node(s) 102, client machine(s)102, client computer(s) 102, client device(s) 102, endpoint(s) 102, orendpoint node(s) 102) in communication with one or more remote machines106 a-106 n (also generally referred to as server(s) 106 or remotemachine(s) 106) via one or more networks 104. In some embodiments, alocal machine 102 has the capacity to function as both a client nodeseeking access to resources provided by a server and as a serverproviding access to hosted resources for other clients 102 a-102 n.

Although FIG. 1A shows a network 104 between the local machines 102 andthe remote machines 106, the local machines 102 and the remote machines106 may be on the same network 104. The network 104 can be a local-areanetwork (LAN), such as a company Intranet, a metropolitan area network(MAN), or a wide area network (WAN), such as the Internet or the WorldWide Web. In some embodiments, there are multiple networks 104 betweenthe local machines 102 and the remote machines 106. In one of theseembodiments, a network 104′ (not shown) may be a private network and anetwork 104 may be a public network. In another of these embodiments, anetwork 104 may be a private network and a network 104′ a publicnetwork. In still another embodiment, networks 104 and 104′ may both beprivate networks. In yet another embodiment, networks 104 and 104′ mayboth be public networks.

The network 104 may be any type and/or form of network and may includeany of the following: a point to point network, a broadcast network, awide area network, a local area network, a telecommunications network, adata communication network, a computer network, an ATM (AsynchronousTransfer Mode) network, a SONET (Synchronous Optical Network) network, aSDH (Synchronous Digital Hierarchy) network, a wireless network and awireline network. In some embodiments, the network 104 may comprise awireless link, such as an infrared channel or satellite band. Thetopology of the network 104 may be a bus, star, or ring networktopology. The network 104 may be of any such network topology as knownto those ordinarily skilled in the art capable of supporting theoperations described herein. The network may comprise mobile telephonenetworks utilizing any protocol or protocols used to communicate amongmobile devices, including AMPS, TDMA, CDMA, GSM, GPRS or UMTS. In someembodiments, different types of data may be transmitted via differentprotocols. In other embodiments, the same types of data may betransmitted via different protocols.

In some embodiments, the system may include multiple, logically-groupedremote machines 106. In one of these embodiments, the logical group ofremote machines may be referred to as a server farm 38. In another ofthese embodiments, the remote machines 106 may be geographicallydispersed. In other embodiments, a server farm 38 may be administered asa single entity. In still other embodiments, the server farm 38comprises a plurality of server farms 38. The remote machines 106 withineach server farm 38 can be heterogeneous—one or more of the remotemachines 106 can operate according to one type of operating systemplatform (e.g., WINDOWS NT, WINDOWS 2003, WINDOWS 2008, WINDOWS 7 andWINDOWS Server 2008 R2, all of which are manufactured by Microsoft Corp.of Redmond, Wash.), while one or more of the other remote machines 106can operate according to another type of operating system platform(e.g., Unix or Linux).

The remote machines 106 of each server farm 8 do not need to bephysically proximate to another remote machine 106 in the same serverfarm 38. Thus, the group of remote machines 106 logically grouped as aserver farm 38 may be interconnected using a wide-area network (WAN)connection or a metropolitan-area network (MAN) connection. For example,a server farm 38 may include remote machines 106 physically located indifferent continents or different regions of a continent, country,state, city, campus, or room. Data transmission speeds between remotemachines 106 in the server farm 38 can be increased if the remotemachines 106 are connected using a local-area network (LAN) connectionor some form of direct connection.

A remote machine 106 may be a file server, application server, webserver, proxy server, appliance, network appliance, gateway, applicationgateway, gateway server, virtualization server, deployment server, SSLVPN server, or firewall. In some embodiments, a remote machine 106provides a remote authentication dial-in user service, and is referredto as a RADIUS server. In other embodiments, a remote machine 106 mayhave the capacity to function as either an application server or as amaster application server. In still other embodiments, a remote machine106 is a blade server. In yet other embodiments, a remote machine 106executes a virtual machine providing, to a user or client computer 102,access to a computing environment.

In one embodiment, a remote machine 106 may include an Active Directory.The remote machine 106 may be an application acceleration appliance. Forembodiments in which the remote machine 106 is an applicationacceleration appliance, the remote machine 106 may provide functionalityincluding firewall functionality, application firewall functionality, orload balancing functionality. In some embodiments, the remote machine106 comprises an appliance such as one of the line of appliancesmanufactured by the Citrix Application Networking Group, of San Jose,Calif., or Silver Peak Systems, Inc., of Mountain View, Calif., or ofRiverbed Technology, Inc., of San Francisco, Calif., or of F5 Networks,Inc., of Seattle, Wash., or of Juniper Networks, Inc., of Sunnyvale,Calif.

In some embodiments, a remote machine 106 executes an application onbehalf of a user of a local machine 102. In other embodiments, a remotemachine 106 executes a virtual machine, which provides an executionsession within which applications execute on behalf of a user of a localmachine 102. In one of these embodiments, the execution session is ahosted desktop session. In another of these embodiments, the executionsession provides access to a computing environment, which may compriseone or more of: an application, a plurality of applications, a desktopapplication, and a desktop session in which one or more applications mayexecute.

In some embodiments, a local machine 102 communicates with a remotemachine 106. In one embodiment, the local machine 102 communicatesdirectly with one of the remote machines 106 in a server farm 38. Inanother embodiment, the local machine 102 executes a programneighborhood application to communicate with a remote machine 106 in aserver farm 38. In still another embodiment, the remote machine 106provides the functionality of a master node. In some embodiments, thelocal machine 102 communicates with the remote machine 106 in the serverfarm 38 through a network 104. Over the network 104, the local machine102 can, for example, request execution of various applications hostedby the remote machines 106 a-106 n in the server farm 38 and receiveoutput of the results of the application execution for display. In someembodiments, only a master node provides the functionality required toidentify and provide address information associated with a remotemachine 106 b hosting a requested application.

In one embodiment, the remote machine 106 provides the functionality ofa web server. In another embodiment, the remote machine 106 a receivesrequests from the local machine 102, forwards the requests to a secondremote machine 106 b and responds to the request by the local machine102 with a response to the request from the remote machine 106 b. Instill another embodiment, the remote machine 106 a acquires anenumeration of applications available to the local machine 102 andaddress information associated with a remote machine 106 b hosting anapplication identified by the enumeration of applications. In yetanother embodiment, the remote machine 106 presents the response to therequest to the local machine 102 using a web interface. In oneembodiment, the local machine 102 communicates directly with the remotemachine 106 to access the identified application. In another embodiment,the local machine 102 receives output data, such as display data,generated by an execution of the identified application on the remotemachine 106.

In some embodiments, the remote machine 106 or a server farm 38 may berunning one or more applications, such as an application providing athin-client computing or remote display presentation application. In oneembodiment, the remote machine 106 or server farm 38 executes as anapplication any portion of the CITRIX ACCESS SUITE by Citrix Systems,Inc., such as the METAFRAME or CITRIX PRESENTATION SERVER products, anyof the following products manufactured by Citrix Systems, Inc.: CITRIXXENAPP, CITRIX XENDESKTOP, CITRIX ACCESS GATEWAY, and/or any of theMICROSOFT WINDOWS Terminal Services manufactured by the MicrosoftCorporation. In another embodiment, the application is an ICA client,developed by Citrix Systems, Inc. of Fort Lauderdale, Fla. In stillanother embodiment, the remote machine 106 may run an application,which, for example, may be an application server providing emailservices such as MICROSOFT EXCHANGE manufactured by the MicrosoftCorporation of Redmond, Wash., a web or Internet server, or a desktopsharing server, or a collaboration server. In yet another embodiment,any of the applications may comprise any type of hosted service orproducts, such as GOTOMEETING provided by Citrix Online Division, Inc.of Santa Barbara, Calif., WEBEX provided by WebEx, Inc. of Santa Clara,Calif., or Microsoft Office LIVE MEETING provided by MicrosoftCorporation of Redmond, Wash.

A local machine 102 may execute, operate or otherwise provide anapplication, which can be any type and/or form of software, program, orexecutable instructions such as any type and/or form of web browser,web-based client, client-server application, a thin-client computingclient, an ActiveX control, or a Java applet, or any other type and/orform of executable instructions capable of executing on local machine102. In some embodiments, the application may be a server-based or aremote-based application executed on behalf of the local machine 102 ona remote machine 106. In other embodiments, the remote machine 106 maydisplay output to the local machine 102 using any thin-client protocol,presentation layer protocol, or remote-display protocol, such as theIndependent Computing Architecture (ICA) protocol manufactured by CitrixSystems, Inc. of Ft. Lauderdale, Fla.; the Remote Desktop Protocol (RDP)manufactured by the Microsoft Corporation of Redmond, Wash.; the XIIprotocol; the Virtual Network Computing (VNC) protocol, manufactured byAT&T Bell Labs; the SPICE protocol, manufactured by Qumranet, Inc., ofSunnyvale, Calif., USA, and of Raanana, Israel; the Net2Displayprotocol, manufactured by VESA, of Milpitas, Calif.; the PC-over-IPprotocol, manufactured by Teradici Corporation, of Burnaby, B.C.; theTCX protocol, manufactured by Wyse Technology, Inc., of San Jose,Calif.; the THINC protocol developed by Columbia University in the Cityof New York, of New York, N.Y.; or the Virtual-D protocols manufacturedby Desktone, Inc., of Chelmsford, Mass. The application can use any typeof protocol and it can be, for example, an HTTP client, an FTP client,an Oscar client, or a Telnet client. In still other embodiments, theapplication comprises any type of software related to voice overInternet protocol (VoiP) communications, such as a soft IP telephone. Infurther embodiments, the application comprises any application relatedto real-time data communications, such as applications for streamingvideo and/or audio.

The local machine 102 and remote machine 106 may be deployed as and/orexecuted on any type and form of computing device, such as a computer,network device or appliance capable of communicating on any type andform of network and performing the operations described herein. FIGS. 1Band 1C depict block diagrams of a computing device 100 useful forpracticing an embodiment of the local machine 102 or a remote machine106. As shown in FIGS. 1B and 1C, each computing device 100 includes acentral processing unit 121, and a main memory unit 122. As shown inFIG. 1B, a computing device 100 may include a storage device 128, aninstallation device 116, a network interface 118, an I/O controller 123,display devices 124 a-n, a keyboard 126 and a pointing device 127, suchas a mouse. The storage device 128 may include, without limitation, anoperating system, software, and a client agent 120. As shown in FIG. 1C,each computing device 100 may also include additional optional elements,such as a memory port 103, a bridge 170, one or more input/outputdevices 130 a-130 n (generally referred to using reference numeral 130),and a cache memory 140 in communication with the central processing unit121.

The central processing unit 121 is any logic circuitry that responds toand processes instructions fetched from the main memory unit 122. Inmany embodiments, the central processing unit 121 is provided by amicroprocessor unit, such as: those manufactured by Intel Corporation ofMountain View, Calif.; those manufactured by Motorola Corporation ofSchaumburg, Ill.; those manufactured by Transmeta Corporation of SantaClara, Calif.; the RS/6000 processor, those manufactured byInternational Business Machines of White Plains, N.Y.; or thosemanufactured by Advanced Micro Devices of Sunnyvale, Calif. Thecomputing device 100 may be based on any of these processors, or anyother processor capable of operating as described herein.

Main memory unit 122 may be one or more memory chips capable of storingdata and allowing any storage location to be directly accessed by themicroprocessor 121, such as Static random access memory (SRAM), BurstSRAM or SynchBurst SRAM (BSRAM), Dynamic random access memory (DRAM),Fast Page Mode DRAM (FPM DRAM), Enhanced DRAM (EDRAM), Extended DataOutput RAM (EDO RAM), Extended Data Output DRAM (EDO DRAM), BurstExtended Data Output DRAM (BEDO DRAM), Enhanced DRAM (EDRAM),synchronous DRAM (SDRAM), JEDEC SRAM, PC100 SDRAM, Double Data RateSDRAM (DDR SDRAM), Enhanced SDRAM (ESDRAM), SyncLink DRAM (SLDRAM),Direct Rambus DRAM (DRDRAM), or Ferroelectric RAM (FRAM). The mainmemory 122 may be based on any of the above described memory chips, orany other available memory chips capable of operating as describedherein. In the embodiment shown in FIG. 1B, the processor 121communicates with main memory 122 via a system bus 150 (described inmore detail below). FIG. 1C depicts an embodiment of a computing device100 in which the processor communicates directly with main memory 122via a memory port 103. For example, in FIG. 1C the main memory 122 maybe DRDRAM.

FIG. 1C depicts an embodiment in which the main processor 121communicates directly with cache memory 140 via a secondary bus,sometimes referred to as a backside bus. In other embodiments, the mainprocessor 121 communicates with cache memory 140 using the system bus150. Cache memory 140 typically has a faster response time than mainmemory 122 and is typically provided by SRAM, BSRAM, or EDRAM. In theembodiment shown in FIG. 1B, the processor 121 communicates with variousI/O devices 130 via a local system bus 150. Various buses may be used toconnect the central processing unit 121 to any of the I/O devices 130,including a VESA VL bus, an ISA bus, an EISA bus, a MicroChannelArchitecture (MCA) bus, a PCI bus, a PCI-X bus, a PCI-Express bus, or aNuBus. For embodiments in which the I/O device is a video display 124,the processor 121 may use an Advanced Graphics Port (AGP) to communicatewith the display 124. FIG. 1C depicts an embodiment of a computer 100 inwhich the main processor 121 communicates directly with I/O device 130 bvia HYPERTRANSPORT, RAPIDIO, or INFINIBAND communications technology.FIG. 1C also depicts an embodiment in which local busses and directcommunication are mixed: the processor 121 communicates with I/O device130 a using a local interconnect bus while communicating with I/O device130 b directly.

A wide variety of I/O devices 130 a-130 n may be present in thecomputing device 100. Input devices include keyboards, mice, trackpads,trackballs, microphones, and drawing tablets. Output devices includevideo displays, speakers, inkjet printers, laser printers, anddye-sublimation printers. An I/O controller 123, as shown in FIG. 1B,may control the I/O devices. The I/O controller may control one or moreI/O devices such as a keyboard 126 and a pointing device 127, e.g., amouse or optical pen. Furthermore, an I/O device may also providestorage and/or an installation medium 116 for the computing device 100.In still other embodiments, the computing device 100 may provide USBconnections (not shown) to receive handheld USB storage devices such asthe USB Flash Drive line of devices manufactured by Twintech Industry,Inc. of Los Alamitos, Calif.

Referring again to FIG. 1B, the computing device 100 may support anysuitable installation device 116, such as a floppy disk drive forreceiving floppy disks such as 3.5-inch, 5.25-inch disks or ZIP disks, aCD-ROM drive, a CD-R/RW drive, a DVD-ROM drive, tape drives of variousformats, USB device, hard-drive or any other device suitable forinstalling software and programs. The computing device 100 may furthercomprise a storage device, such as one or more hard disk drives orredundant arrays of independent disks, for storing an operating systemand other related software, and for storing application softwareprograms such as any program related to the client agent 120.Optionally, any of the installation devices 116 could also be used asthe storage device. Additionally, the operating system and the softwarecan be run from a bootable medium, for example, a bootable CD, such asKNOPPIX, a bootable CD for GNU/Linux that is available as a GNU/Linuxdistribution from knoppix.net.

Furthermore, the computing device 100 may include a network interface118 to interface to the network 104 through a variety of connectionsincluding, but not limited to, standard telephone lines, LAN or WANlinks (e.g., 802.11, T1, T3, 56 kb, X.25, SNA, DECNET), broadbandconnections (e.g., ISDN, Frame Relay, ATM, Gigabit Ethernet,Ethernet-over-SONET), wireless connections, or some combination of anyor all of the above. Connections can be established using a variety ofcommunication protocols (e.g., TCP/IP, IPX, SPX, NetBIOS, Ethernet,ARCNET, SONET, SDH, Fiber Distributed Data Interface (FDDI), RS232, IEEE802.11, IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, CDMA, GSM, WiMax anddirect asynchronous connections). In one embodiment, the computingdevice 100 communicates with other computing devices 100′ via any typeand/or form of gateway or tunneling protocol such as Secure Socket Layer(SSL) or Transport Layer Security (TLS), or the Citrix Gateway Protocolmanufactured by Citrix Systems, Inc. of Ft. Lauderdale, Fla. The networkinterface 118 may comprise a built-in network adapter, network interfacecard, PCMCIA network card, card bus network adapter, wireless networkadapter, USB network adapter, modern or any other device suitable forinterfacing the computing device 100 to any type of network capable ofcommunication and performing the operations described herein.

In some embodiments, the computing device 100 may comprise or beconnected to multiple display devices 124 a-124 n, which each may be ofthe same or different type and/or form. As such, any of the I/O devices130 a-130 n and/or the I/O controller 123 may comprise any type and/orform of suitable hardware, software, or combination of hardware andsoftware to support, enable or provide for the connection and use ofmultiple display devices 124 a-124 n by the computing device 100. Forexample, the computing device 100 may include any type and/or form ofvideo adapter, video card, driver, and/or library to interface,communicate, connect or otherwise use the display devices 124 a-124 n.In one embodiment, a video adapter may comprise multiple connectors tointerface to multiple display devices 124 a-124 n. In other embodiments,the computing device 100 may include multiple video adapters, with eachvideo adapter connected to one or more of the display devices 124 a-124n. In some embodiments, any portion of the operating system of thecomputing device 100 may be configured for using multiple displays 124a-124 n. In other embodiments, one or more of the display devices 124a-124 n may be provided by one or more other computing devices, such ascomputing devices 100 a and 100 b connected to the computing device 100,for example, via a network. These embodiments may include any type ofsoftware designed and constructed to use another computer's displaydevice as a second display device 124 a for the computing device 100.One ordinarily skilled in the art will recognize and appreciate thevarious ways and embodiments that a computing device 100 may beconfigured to have multiple display devices 124 a-124 n.

In further embodiments, an I/O device 130 may be a bridge between thesystem bus 150 and an external communication bus, such as a USB bus, anApple Desktop Bus, an RS-232 serial connection, a SCSI bus, a FireWirebus, a FireWire 800 bus, an Ethernet bus, an AppleTalk bus, a GigabitEthernet bus, an Asynchronous Transfer Mode bus, a HIPPI bus, a SuperHIPPI bus, a SerialPlus bus, a SCI/LAMP bus, a FibreChannel bus, or aSerial Attached small computer system interface bus.

A computing device 100 of the sort depicted in FIGS. 1B and 1C typicallyoperates under the control of operating systems, which controlscheduling of tasks and access to system resources. The computing device100 can be running any operating system such as any of the versions ofthe MICROSOFT WINDOWS operating systems, the different releases of theUnix and Linux operating systems, any version of the MAC OS forMacintosh computers, any embedded operating system, any real-timeoperating system, any open source operating system, any proprietaryoperating system, any operating systems for mobile computing devices, orany other operating system capable of running on the computing deviceand performing the operations described herein. Typical operatingsystems include, but are not limited to: WINDOWS 3.x, WINDOWS 95,WINDOWS 98, WINDOWS 2000, WINDOWS NT 3.51, WINDOWS NT 4.0, WINDOWS 7,WINDOWS CE, WINDOWS XP, and WINDOWS VISTA, all of which are manufacturedby Microsoft Corporation of Redmond, Wash.; MAC OS, manufactured byApple Inc., of Cupertino, Calif.; OS/2, manufactured by InternationalBusiness Machines of Armonk, N.Y.; and Linux, a freely-availableoperating system distributed by Caldera Corp. of Salt Lake City, Utah,or any type and/or form of a Unix operating system, among others.

The computing device 100 can be any workstation, desktop computer,laptop or notebook computer, server, handheld computer, mobile telephoneor other portable telecommunication device, media playing device, agaming system, mobile computing device, or any other type and/or form ofcomputing, telecommunications or media device that is capable ofcommunication and that has sufficient processor power and memorycapacity to perform the operations described herein. For example, thecomputing device 100 may comprise a device of the IPOD family of devicesmanufactured by Apple Inc., of Cupertino, Calif., a PLAYSTATION 2,PLAYSTATION 3, or PERSONAL PLAYSTATION PORTABLE (PSP) devicemanufactured by the Sony Corporation of Tokyo, Japan, a NINTENDO DS,NINTENDO GAMEBOY, NINTENDO GAMEBOY ADVANCED or NINTENDO REVOLUTIONdevice manufactured by Nintendo Co., Ltd., of Kyoto, Japan, or an XBOXor XBOX 360 device manufactured by the Microsoft Corporation of Redmond,Wash.

In some embodiments, the computing device 100 may have differentprocessors, operating systems, and input devices consistent with thedevice. For example, in one embodiment, the computing device 100 is aTREO 180, 270, 600, 650, 680, 700p, 700w/wx, 750, 755p, 800w, Centro, orPro smart phone manufactured by Palm, Inc. In some of these embodiments,the TREO smart phone is operated under the control of the PalmOSoperating system and includes a stylus input device as well as afive-way navigator device.

In other embodiments the computing device 100 is a mobile device, suchas a JAVA-enabled cellular telephone or personal digital assistant(PDA), such as the i55sr, i58sr, i85s, i88s, i90c, i95cl, i335, i365,i570, I576, i580, i615, i760, i836, i850, i870, i880, i920, i930, ic502,ic602, ic902, i776 or the im1100, all of which are manufactured byMotorola Corp. of Schaumburg, Ill., the 6035 or the 7135, manufacturedby Kyocera of Kyoto, Japan, or the i300 or i330, manufactured by SamsungElectronics Co., Ltd., of Seoul, Korea. In some embodiments, thecomputing device 100 is a mobile device manufactured by Nokia ofFinland, or by Sony Ericsson Mobile Communications AB of Lund, Sweden.

In still other embodiments, the computing device 100 is a Blackberryhandheld or smart phone, such as the devices manufactured by Research InMotion Limited, including the Blackberry 7100 series, 8700 series, 7700series, 7200 series, the Blackberry 7520, the Blackberry PEARL 8100, the8700 series, the 8800 series, the Blackberry Storm, Blackberry Bold,Blackberry Curve 8900, and the Blackberry Pearl Flip. In yet otherembodiments, the computing device 100 is a smart phone, Pocket PC,Pocket PC Phone, or other handheld mobile device supporting MicrosoftWindows Mobile Software. Moreover, the computing device 100 can be anyworkstation, desktop computer, laptop or notebook computer, server,handheld computer, mobile telephone, any other computer, or other formof computing or telecommunications device that is capable ofcommunication and that has sufficient processor power and memorycapacity to perform the operations described herein.

In some embodiments, the computing device 100 is a digital audio player.In one of these embodiments, the computing device 100 is a digital audioplayer such as the Apple IPOD, IPOD Touch, IPOD NANO, and IPOD SHUFFLElines of devices, manufactured by Apple Inc., of Cupertino, Calif. Inanother of these embodiments, the digital audio player may function asboth a portable media player and as a mass storage device. In otherembodiments, the computing device 100 is a digital audio player such asthe DigitalAudioPlayer Select MP3 players, manufactured by SamsungElectronics America, of Ridgefield Park, N.J., or the Motorola m500 orm25 Digital Audio Players, manufactured by Motorola Inc. of Schaumburg,Ill. In still other embodiments, the computing device 100 is a portablemedia player, such as the Zen Vision W, the Zen Vision series, the ZenPortable Media Center devices, or the Digital MP3 line of MP3 players,manufactured by Creative Technologies Ltd. In yet other embodiments, thecomputing device 100 is a portable media player or digital audio playersupporting file formats including, but not limited to, MP3, WAV,M4A/AAC, WMA Protected AAC, AIFF, Audible audiobook, Apple Losslessaudio file formats and .mov, .m4v, and .mp4 MPEG-4 (H.264/MPEG-4 AVC)video file formats.

In some embodiments, the computing device 100 comprises a combination ofdevices, such as a mobile phone combined with a digital audio player orportable media player. In one of these embodiments, the computing device100 is a Motorola RAZR or Motorola ROKR line of combination digitalaudio players and mobile phones. In another of these embodiments, thecomputing device 100 is a device in the iPhone line of smartphones,manufactured by Apple Inc., of Cupertino, Calif.

In one embodiment, a computing device 102 a may request resources from aremote machine 106, while providing the functionality of a remotemachine 106 to a client 102 b. In such an embodiment, the computingdevice 102 a may be referred to as a client with respect to datareceived from the remote machine 106 (which may be referred to as aserver) and the computing device 102 a may be referred to as a serverwith respect to the second client 102 b. In another embodiment, theclient 102 may request resources from the remote machine 106 on behalfof a user of the client 102.

As shown in FIG. 1D, the computing device 100 may comprise multipleprocessors and may provide functionality for simultaneous execution ofinstructions or for simultaneous execution of one instruction on morethan one piece of data. In some embodiments, the computing device 100may comprise a parallel processor with one or more cores. In one ofthese embodiments, the computing device 100 is a shared memory paralleldevice, with multiple processors and/or multiple processor cores,accessing all available memory as a single global address space. Inanother of these embodiments, the computing device 100 is a distributedmemory parallel device with multiple processors each accessing localmemory only. In still another of these embodiments, the computing device100 has both some memory which is shared and some memory which can onlybe accessed by particular processors or subsets of processors. In stilleven another of these embodiments, the computing device 100, such as amulticore microprocessor, combines two or more independent processorsinto a single package, often a single integrated circuit (IC). In yetanother of these embodiments, the computing device 100 includes a chiphaving a CELL BROADBAND ENGINE architecture and including a Powerprocessor element and a plurality of synergistic processing elements,the Power processor element and the plurality of synergistic processingelements linked together by an internal high speed bus, which may bereferred to as an element interconnect bus.

In some embodiments, the processors provide functionality for executionof a single instruction simultaneously on multiple pieces of data(SIMD). In other embodiments, the processors provide functionality forexecution of multiple instructions simultaneously on multiple pieces ofdata (MIMD). In still other embodiments, the processor may use anycombination of SIMD and MIMD cores in a single device.

In some embodiments, the computing device 100 may comprise a graphicsprocessing unit. In one of these embodiments, depicted in FIG. 1E, thecomputing device 100 includes at least one central processing unit 121and at least one graphics processing unit. In another of theseembodiments, the computing device 100 includes at least one parallelprocessing unit and at least one graphics processing unit. In stillanother of these embodiments, the computing device 100 includes aplurality of processing units of any type, one of the plurality ofprocessing units comprising a graphics processing unit.

In one embodiment, a resource may be a program, an application, adocument, a file, a plurality of applications, a plurality of files, anexecutable program file, a desktop environment, a computing environment,or other resource made available to a user of the local computing device102. The resource may be delivered to the local computing device 102 viaa plurality of access methods including, but not limited to,conventional installation directly on the local computing device 102,delivery to the local computing device 102 via a method for applicationstreaming, delivery to the local computing device 102 of output datagenerated by an execution of the resource on a third computing device106 b and communicated to the local computing device 102 via apresentation layer protocol, delivery to the local computing device 102of output data generated by an execution of the resource via a virtualmachine executing on a remote computing device 106, or execution from aremovable storage device connected to the local computing device 102,such as a USB device, or via a virtual machine executing on the localcomputing device 102 and generating output data. In some embodiments,the local computing device 102 transmits output data generated by theexecution of the resource to another client computing device 102 b.

In some embodiments, a user of a local computing device 102 connects toa remote computing device 106 and views a display on the local computingdevice 102 of a local version of a remote desktop environment,comprising a plurality of data objects, generated on the remotecomputing device 106. In one of these embodiments, at least one resourceis provided to the user by the remote computing device 106 (or by asecond remote computing device 106 b) and displayed in the remotedesktop environment. However, there may be resources that the userexecutes on the local computing device 102, either by choice, or due toa policy or technological requirement. In another of these embodiments,the user of the local computing device 102 would prefer an integrateddesktop environment providing access to all of the resources availableto the user, instead of separate desktop environments for resourcesprovided by separate machines. For example, a user may find navigatingbetween multiple graphical displays confusing and difficult to useproductively. Or, a user may wish to use the data generated by oneapplication provided by one machine in conjunction with another resourceprovided by a different machine. In still another of these embodiments,requests for execution of a resource, windowing moves, applicationminimize/maximize, resizing windows, and termination of executingresources may be controlled by interacting with a remote desktopenvironment that integrates the display of the remote resources and ofthe local resources. In yet another of these embodiments, an applicationor other resource accessible via an integrated desktopenvironment—including those resources executed on the local computingdevice 102 and those executed on the remote computing device 106—isshown in a single desktop environment.

In one embodiment, data objects from a remote computing device 106 areintegrated into a desktop environment generated by the local computingdevice 102. In another embodiment, the remote computing device 106maintains the integrated desktop. In still another embodiment, the localcomputing device 102 maintains the integrated desktop.

In some embodiments, a single remote desktop environment 204 isdisplayed. In one of these embodiments, the remote desktop environment204 is displayed as a full-screen desktop. In other embodiments, aplurality of remote desktop environments 204 is displayed. In one ofthese embodiments, one or more of the remote desktop environments aredisplayed in non-full-screen mode on one or more display devices 124. Inanother of these embodiments, the remote desktop environments aredisplayed in full-screen mode on individual display devices. In stillanother of these embodiments, one or more of the remote desktopenvironments are displayed in full-screen mode on one or more displaydevices 124.

Section B. Systems and Methods for Integrating Local and RemoteApplications and Desktops to Provide a Seamless User Experience.

Referring now to FIG. 2, a block diagram depicts one embodiment of asystem for displaying, in a user interface element generated anddisplayed by a local machine, an identification of graphical datagenerated and displayed on the local machine and an identification ofgraphical data generated on a remote machine and displayed on the localmachine. In brief overview, the system 200 includes a first agent 202executing on a local computing device 102, a second agent 204 executingon a remote computing device 106, a first process 206 executing on theremote computing device 106, and a second process 218 executing on thelocal computing device 102. The first agent 202 receives, from thesecond agent 204, an identifier of the first process 206 and anidentification of a first window 207 generated by the first process 206.The first agent 202 associates a second window 212 with the identifierof the first process 206, the second window 212 generated by the firstagent 202 on the local machine 102. Responsive to the association of thesecond window 212 with the identifier of the first process 206, a shell214 executing on the local machine 102 displays, in a taskbar buttongroup 230, i) an identification of the second window 212 and ii) anidentification of a third window 216, the third window 216 generated bythe second process 218 and displayed on the local machine 102.

In some embodiments, a process executing on a computing device—such asthe first process 206 executing on the remote computing device 106 orthe second process 218 executing on the local computing device102—generates output data and window attribute data and communicateswith a shell executing on the computing device to display the outputdata according to the window attribute data. In some embodiments, thisfirst process 206 may also be referred to as a remote application. Inother embodiments, the first agent 202 receives graphical data andwindow attribute data from the second agent 204 and directs the displayof the received graphical or window attribute data in a desktopenvironment including a plurality of data objects. In one of theseembodiments, a data object is a window displayed in the desktopenvironment. In another one of these embodiments, the data object is adata structure storing attribute data and may or may not have anassociated visible representation in the desktop environment. In stillanother of these embodiments, a data object is a data structure storingdata associated with a user interface element—visual state,identification of associated functionality, location of graphical data,title bar contents, etc.—and a window is a graphical representation ofthe user interface element. In still even another of these embodiments,a shell 214 executing on a machine provides a display of user interfaceelements in a desktop environment. This shell may be referred tovariously as a finder, a graphical user interface (GUI), a windows orX-windows interface, or any other similar term. In another of theseembodiments, the shell 214 displays graphical data associated with adata object in accordance with attribute data associated with the dataobject. In yet another of these embodiments, the first agent 202communicates with the shell 214 to direct the local display of remotelygenerated data.

Referring now to FIG. 2, and in greater detail, the first agent 202executes on the local computing device 102. Although referred to as afirst agent, in some embodiments, first agent 202 may be referred to asa local client, local client process, local client agent, or any othersimilar term. In one embodiment, the local computing device is acomputing device as described above in connection with FIG. 1A-1E. Inanother embodiment, the local computing device is a client device 102,connecting to a server 106 to access one or more resources available toa user of the local computing device 102. In still another embodiment,the first agent 202 is part of a presentation layer protocol agent. Inyet another embodiment, the first agent 202 is in communication with apresentation layer protocol agent.

The second agent 204 executes on the remote computing device 106. Aswith the first agent, in some embodiments, the second agent may bereferred to as a remote agent, a remote client, a remote process, aserver agent, a server process, or any other similar term. In oneembodiment, the remote computing device is a computing device asdescribed above in connection with FIG. 1A-1E. In another embodiment,the second agent 204 is part of a presentation layer protocol agent. Instill another embodiment, the second agent 204 is in communication witha presentation layer protocol agent.

In some embodiments, the first agent 202 includes a receiver forreceiving, from the second agent 204, data associated with a desktopenvironment generated on the remote machine 106. In one of theseembodiments, for example, the first agent 202 includes a receiver—whichmay be provided as, by way of example, a dynamically linked librarycomponent—that receives window creation and window process data from thesecond agent 204 for use in displaying a local version of a windowgenerated on the remote machine 106. In some embodiments, the firstagent 202 may receive data, such as output data 208 and window attributedata 210 over one or more connections. In one embodiment, one or moreconnections may be multiplexed into one or more virtual channels. Suchmultiplexing may allow for different virtual channels to have differentbandwidth limits or different priorities, while still being part of asingle transport layer connection. This may reduce the transport layeroverhead required and provide for SSL or VPN tunnel capability, whilestill allowing per-channel compression, buffering, and management ofcommunication priority between second agent 204 and first agent 202. Insome embodiments, such virtual channels may be dedicated to specificcontent types or purposes. For example, a first high-priority virtualchannel may be dedicated to transmission of output data 208, while asecond low-priority virtual channel may be dedicated to transmission oftaskbar thumbnail images, discussed in more detail below. In someembodiments, virtual channels may be opened or closed without needing todisestablish or reestablish the transport layer connection over whichthey communicate.

In one embodiment, the shell 214 is software providing a user interfaceto the user of a computing device. In one embodiment, a shell may besupplemented or replaced with a third-party shell. In MICROSOFT WINDOWS,the default shell is EXPLORER, which determines the configuration of thedesktop (e.g., the task bar, notification area, start menu, etc.).Although referred to as a shell, as discussed above, the shell may alsobe referred to as a graphical user interface or GUI, a finder, anexplorer, a windows interface, or any other similar term.

In some embodiments, the first agent 202 includes functionality forcommunicating with the shell 214 to modify a display of the desktop. Inone of these embodiments, the first agent 202 includes a transmittersending instructions to a component in the operating system thatgenerates and maintains a display of data in the desktop environment. Inanother of these embodiments, the first agent 202 includes a componentthat provides the first agent 202 with functionality for storing windowattribute data or transmitting display instructions to the operatingsystem; for example, the first agent 202 may include adynamically-linked library component for maintaining or modifyingtaskbar data. In some embodiments, the transmitter is in communicationwith a receiver in the first agent 202 that receives window attributedata 210 and output data 208 from the second agent 204. In one of theseembodiments, the receiver within the first agent 202 receives data fromthe second agent 204 and forwards the received data to the transmitter,which sends instructions to the operating system based upon theforwarded data. In other embodiments, the first agent 202 includes acomponent for storing data received from the second agent 204, such as,by way of example, window attribute data.

In some embodiments, window attribute data 210 or output data 208 maycomprise an icon representative of the first window 207 or first process206. In another embodiment, window attribute data 210 or output data 208may comprise an icon of the application or process that generated thewindow. In many embodiments, the first agent 202 may receive an icon orbitmap of an icon of the first process 206 or first window 207 fordisplay within a taskbar 226 or other user interface element as a localdisplay of window attribute data 210. Accordingly, when a taskbar buttongroup is interacted with by a user of local computing device 102, thetaskbar button group may display the received icon and/or textcomprising the title of second window 212, the first window 207, orfirst process 206. Referring ahead briefly, an example screenshot of onesuch embodiment is shown in FIG. 4A, illustrating display of a notepadicon of a remote process. As shown in FIG. 4A, in these embodiments, thetaskbar button group may display an icon for a remote application and athumbnail for a local application.

Referring back to FIG. 2, in another embodiment, first agent 202 mayreceive a static screenshot or bitmap of output data of a first window,for display in a taskbar button group. In some embodiments, suchscreenshot or bitmap may be reduced in scale. For example, output datamay comprise a 400 by 400 pixel window, but second agent 204 may send a40 by 40 pixel thumbnail for display in the taskbar button group. Thismay reduce bandwidth requirements. Such static screenshots may be sentperiodically, or responsive to user commands. For example, in oneembodiment, first agent 202 may detect user interaction with the taskbarbutton group, interaction with a 3D or flip-3d interface, or input of analt-tab or similar command. Responsive to detection of such interaction,in one embodiment, the first agent 202 may request a refresh of outputdata 208 of a window or request redraw of output data 208, receive suchrefreshed or redrawn output data 208, and may display a thumbnail of thenewly received output data in the taskbar button group or other userinterface element. In another embodiment, responsive to detection of theinteraction, the first agent 202 may request a new, redrawn, orrefreshed static thumbnail of the output data as discussed above.

In some embodiments, taskbar 226 may comprise functionality fordisplaying either an icon of an application, or a thumbnail image. Insome embodiments, such thumbnail image may be rendered by taskbar 226from the contents of a local window, while in other embodiments, thethumbnail image may be generated by another element, such as a shell 214or local desktop environment 220, or may be retrieved from a memoryelement. Similar to this latter option, an application icon may bestored as a bitmap in a memory element, and taskbar 226 may retrieve theapplication icon from the memory element for display. Accordingly, inone embodiment, taskbar 226 may comprise functionality for retrieving animage or bitmap from a memory element and displaying the image orbitmap, agnostic to whether the image or bitmap is an icon or thumbnail.Described another way, a thumbnail image of window output may be storedas if it were an application icon, and taskbar 226 may be directed todisplay the thumbnail image as if it were any other application icon.This may allow for display of thumbnail images on legacy systems thatonly have the capability of displaying application icons.

Some versions of operating systems utilizing a taskbar may use one ormore identifiers to group buttons in the taskbar. For example, Windows7, manufactured by the Microsoft Corporation, uses AppIDs set for eachwindow to determine how to group taskbar buttons corresponding to eachwindow. In some embodiments, these AppIDs may be explicitly set by anapplication manufacturer. For example, the AppID for Microsoft Word maybe explicitly set by Microsoft. When the operating system detects twotaskbar buttons with an AppID corresponding to Microsoft Word, theoperating system may group these buttons into a single taskbar buttongroup. In other embodiments, AppIDs may be implicitly set. One suchmethod involves the file system path to the process that created thewindow. For example, if an application is at C:\Program Files\MyCompany\My Application.exe, then the system may translate this filesystem path into a string to use as the AppID. If the applicationgenerates multiple windows, they will each have identical AppIDs, andmay be appropriately grouped.

Some other versions of operating systems use just the file system pathfor taskbar button grouping. For example, Windows XP or Windows Vista,also by Microsoft Corporation, use just the latter method discussedabove of file system paths to determine taskbar button grouping. Thispresents two difficulties with local display of application output fromremote applications. First, the remote application that initiallygenerated a window may have a different file system path from the localclient, particularly with server-side virtualization techniques. Second,the local client may generate a window for application output, and thusthe operating system may consider the path to the local client to be theproper file system path. Thus, if the local client generates two windowsfor two different remote applications, the operating system mayinterpret them as both being associated with the local clientapplication, and thereby present them within the same taskbar group,despite them representing two distinct remote applications.

To remedy this, in some embodiments, the remote application may sendremote window configuration information including the application's filesystem path. The local client may modify this file system path byreplacing a portion of the path with a predetermined local path. Forexample, the remote application may be located at D:\ApplicationServer\Remote Applications\Program Files\My Company\My Application.exe.Upon receipt, the local client may modify this path to replace the firstportion, up to “Program Files”, for example, with a globally uniqueidentifier referring to the local system drive and path to thecorresponding Program Files folder. This new file system path may thuscomprise a combination of a local path and a remote path, and thus maybe referred to as a hybrid file system path.

Referring briefly to the mechanics of taskbar grouping, differentoperating systems use different mechanisms for grouping taskbar buttons.For example, in many embodiments, Windows 7, discussed above, allowsarbitrary grouping of taskbar buttons through associations with groups.However, Windows XP and Vista, among other operating systems, use a listrepresenting the taskbar, with entries tagged as button groupsseparating entries representing a button. For example, if a listincludes “Group 1, Button A, Group 2, Button B, Group 3, Button C,Button D, Button E,” there will be three groups, the first two with onebutton each, and the third with three buttons. By default, the systemmay be configured to hide from display in the taskbar button groups withone button, and hide buttons in a group with a plurality of buttons.With these hidden entries not shown, the taskbar button group abovewould show as “Button A, Button B, Group 3,” with the button for Group 3representing three active windows.

In some embodiments, moving a button from one group to another may beperformed by editing this list or changing the association of a buttonand group. In one embodiment, the local client may generate a new windowfor application output. In some embodiments, this new window may becreated as part of the button group corresponding to the local client.The local client may determine a taskbar button group identifier for thenew window using any of the methods discussed above. In someembodiments, the local client may search the taskbar to determine if anexisting button group exists that comprises a similar identifier. Forexample, if a button group already exists for a notepad application, andthe new window has a taskbar button group identifier corresponding to anotepad application, the local client may determine that a proper buttongroup already exists. The local client may then move this button entryin the taskbar list to be within the button group. If the local clientdetermines that no proper button group exists (for example, if nocorresponding application is running locally or if no button group hasbeen created for another window of the same remote application), thelocal client may create a new button group in the taskbar list based onthe taskbar button group identifier, and move the button correspondingto the new window to this newly created button group.

Transparent UI Integration Between Local and Remote Applications andDesktops

In some embodiments, it may be desirable to centrally manage, configure,and provide or publish even applications that will be executed on alocal client rather than via a remote session. This allows a user totake advantage of local processing power while still allowingadministrators to centrally manage licensing and configuration. Forexample, an administrator can configure and publish a CAD applicationwhich may be executed on a local computer to take advantage of the localprocessor without incurring network delays. Publishing may be performed,in some embodiments, either by an admin console UI, graphics UI, orlow-level software development kit (SDK).

In some embodiments, a published application administration system mayinclude a command line interface for launching and managing publishedapplications. Such command line interface may include optionalparameters for the published application and/or its working directory.In many embodiments, environment variables may also be used in thecommand line and working directory. The environment variables may beevaluated from the client that launches the published application. Insome embodiments, the command line may support any number of arbitraryparameters.

In some embodiments, published applications and file type associationsmay be preconfigured at time of publishing. When an administratorconfigures an application to be provided to and/or executed locally on aclient, the administrator or publishing system may configure file typeassociations with the published application such that initiating launchof a type of document triggers launch of the specific associatedapplication. Similarly, an icon or icons associated with the applicationmay be provided to the remote system for display in shortcuts, a startmenu, or other elements. Similarly, in some embodiments, a user of alocal client may self-publish an application. Self-publishing maycomprise the user selecting a locally installed application to executewhile being integrated with a live-in desktop or other remote desktop orvirtual machine display, as discussed above. In many of theseembodiments, when self-publishing, the client may provide the remotesystem with icons and file type associations with the locally installedapplication, such that the remote system may include the icons within UIelements such as the start menu or taskbar generated by the remotesystem, and may use the file type associations to trigger launch of thelocally-installed application through selection of associated files ordocuments.

In another embodiment, icons and file type associations for anapplication may be enumerated at run time of the application. These maybe sent from the client to the remote system via a virtual channel, oras part of an already established communications channel. This may bedone to provide both control and convenience to the administrator, suchas when file type associations need to be arbitrated between differentapps, such as a local browser and a remote browser. When publishedapplication icons and file type associations are preconfigured atpublishing time, they may be explicitly set, or enumerated from atrusted virtual desktop appliance which has the apps installed, orenumerated from a sample client machine, or obtained in any othersimilar manner.

In some embodiments, by providing icons for published applications tothe remote system, these published applications may appear in the startmenu, dock, desktop shortcuts, or other application launching userinterface generated by the remote system. In some embodiments, if apublished application is not available at the connected client, it maybe presented as unavailable, for example by graying out the icon, bydrawing an X through the icon, by showing a smaller standardunavailable-resource overlay icon (such as a circle with a diagonalslash mark) in the lower right corner of the application icon, or viasome other similar indicator.

In some embodiments, clicking on a published application icon on theremote desktop or application launcher may trigger the launch of theapplication on the local client. In other embodiments, a publishedapplication may be launched using a command line interface in thevirtual desktop appliance session. The host of the remote session mayperform security validation on both the published application and theparameters passed to the application, if any, during host-to-clientlaunches. In some embodiments, shortcuts may be locked-down and/orcontents encrypted to prevent modifications and requests to launcharbitrary processes at the client. Since the client may not necessarilytrust the server, there may be a separate client-side securitypolicy/lock down in some embodiments.

In some embodiments, client hosted applications may comprise nativelyinstalled applications, streamed/offline applications, AppV virtualizedapplications, ICA or RDP displayed remote applications, or applicationsunder different operating systems. In a further embodiment, whenlaunched from the client the configuration context of these apps may bemodified accordingly.

Referring briefly to FIG. 3B, illustrated is a flow diagram of anembodiment of a method 350 for enumerating published applications andredirecting application initiation requests to a local machine. In briefoverview, at step 352, a remote desktop or virtual desktop appliance,host, or server may receive an enumeration of one or more locallyexecuted or installed applications. At step 354, the remote desktop orvirtual desktop appliance, host, or server may provide an integratedenumeration of locally and remotely executed or installed applications.At step 356, the remote desktop or virtual desktop appliance, host, orserver may receive a request to launch a locally or remotely executedapplication. At step 358, responsive to the request, the remote desktopor virtual desktop appliance, host, or server may redirect the requestto one of either a local shell or a remote shell to initiate launch ofthe application.

Transparent Integration of UI Elements

While using local and remote applications and desktops, the userinterfaces (UIs) generated by the local and remote applications desktopsand presented to the user are often inconsistent and have different lookand feel. For example, in some embodiments, a user of a first operatingsystem, such as Apple's Mac OSX operating system, may view a remoteapplication provided by a second operating system, such as Microsoft,Inc.'s Windows operating system. As a result, application windows andinterface elements may have inconsistent window styles, buttons,scrollbars, etc. In addition, sometimes user interface elements areobtrusive, steal focus, do not match the language of the UIs they areintegrated into. Accordingly, it may be preferable in some embodimentsto seamlessly integrate server-generated UI elements with the client byusing the client's look and feel. In other embodiments, it may bepreferable to seamlessly integrate client-generated UI with the remotedesktop by using the server's look and feel. Selection between these twoembodiments may dynamically change, as a window or desktop is madefull-screen, for example.

In one embodiment, a client may send information to a server regardingsystem metrics, system colors, and code page for encoding text data,which the server applies in the remote session. This information mayalso include visual themes or other elements. In one embodiment, theclient's operating system may draw the title bar and borders of a localwindow displaying graphical output data of a remote application,allowing consistent appearance between this window and other windowsgenerated by the client system. In a further embodiment, other elementsof the local window displaying graphical output data of the remoteapplication may still be provided by the remote operating system orcopied from a logical video buffer (LVB), such as a menu bar and othernon-client regions.

In a still further embodiment, all UI elements may be generated eitherlocally or remotely for a consistent appearance. These elements mayinclude text controls, buttons, progress bars, radio buttons, listboxes, or any other type and form of user interface element. Theseelements may be then presented with the receiver's product andOS-specific UI, and returning status back to the sender to achieve amore unified and transparent UI integration. Depending on the use case,the sender and the receiver could be either the client or the server,such as either local or remote desktops and applications.

Server-to-Client UI Redirection

In many remote desktop systems, when a user of a local client logs on toa remote desktop server, status messages are displayed to the user.During the first stage, while the local client is connecting the remotedesktop server, a first set of status messages may be displayed by anapplication or agent on the local machine. Such first set of statusmessages may include messages indicating that a connection is beingestablished, security credentials are being verified, etc. Afterconnecting, a second stage may occur during which the user's profile isbeing loaded, a virtual machine or desktop is being initialized, etc. Asecond set of status messages may be generated and displayed by theremote desktop's operating system or application, and provided to theclient via the remote desktop. Where the remote desktop is provided by adifferent operating system than the user's operating system, the UI ofthese second set of messages can appear inconsistent to the first set ofmessages generated by the user's operating system, as discussed above.

Accordingly, in one embodiment, the text of the second set of statusmessages may be intercepted and redirected to the client, which maypresent the status messages to the user in a unified fashion by using aclient-side application, agent, or library component to present themessages in a uniform way, consistent with the first set of statusmessages presented during the first stage of the connection. These maybe less obtrusive to the user.

In some embodiments, the first and/or second set of status messages mayinclude:

-   -   Please wait . . .    -   Please wait for the Group Policy Client . . .    -   Please wait for the Local Session Manager . . .    -   Welcome    -   Preparing your desktop . . .    -   Please wait for the User Profile Service . . .    -   Please wait for the Group Policy Client . . .    -   Please wait for the Local Session Manager . . .    -   Please wait for the System Event Notification Service . . .    -   Preparing your desktop . . .

However, in other embodiments, other messages may be generated anddisplayed for the user. In one embodiment, only the string portion ofthe user interface, such as the welcome text, may be communicated to theclient. In a further embodiment, no status or acknowledgement may bereturned back to the server.

Referring briefly to FIG. 3C, illustrated is a flow diagram of anembodiment of a method 360 for displaying remote status messages in alocal format. In brief overview, at step 362, a host virtual or remotedesktop may receive a request from a client to initiate a remote desktopsession. At step 364, the host may intercept or hook a status messagecreated by an authentication component. At step 366, the host maytransmit the status message, the text of the message, and/or resourceidentifiers or other identifiers of the status message to the client forlocal display. At step 368, the client may generate and display thestatus message locally, using graphics components of the client.

In many embodiments, the above-discussed techniques may be applied tomore sophisticated UI redirection, involving individual elements of acomplex UI, as well as both asynchronous and synchronous redirection andreturning individual results back to the sender.

Licensing Annoyance and Licensing Error Messages

In some embodiments, licensing-related UI can be redirected to theclient. Such licensing-related UI may include messages regardingexpiration of licenses, authentication of licenses, purchase of extendedlicenses, etc Annoyance messages refer both to requests for licenseinformation and notification of an expiration of a license or animpending expiration, as well as other non-error related statusinformation. Rather than generating such messages at the server, aserver agent can intercept these messages and send the text stringand/or other information related to these messages to a client, whichmay then generate a message, dialog box, or other UI element in aconsistent style to other local OS-generated UI elements.

Session Reconnection Related UI

In many embodiments, a user may disconnect from a remote session andreconnect to the same remote session without the server needing tore-initialize the virtual machine or desktop. This may be done forroaming purposes, such as where a user working on a remotely hosteddesktop on a thin client or desktop machine switches to a laptop, andwishes to reconnect to the existing remote session without pausing thevirtual machine or desktop, or needing to close and re-launchapplications. In such embodiments, using the message redirectiontechniques discussed herein, the server-based messages regarding statusof the existing session may be redirected to the client operatingsystem, whether on the desktop machine or laptop, such that the clientoperating system may present the UI element natively

Shadowing Prompt and Indicator UI

Shadowing, in many embodiments, refers to the ability of systemadministrators to connect to a user's remote desktop, virtual machine,or virtual desktop session. This may be done for maintenance or teachingpurposes, monitoring purposes, or other reasons. Such connections may beviewing-only, or may provide the administrator the ability to move amouse cursor, execute applications, and input data. In many embodiments,when a session is shadowed by a secondary user or administrator, thefirst user is presented with a user interface element that indicatesshadowing is taking place, such as a system tray pop-up or indicator orother element. In some embodiments, the first user may be presented witha dialog box or other user interface element when shadowing isrequested, such that the user may grant permission for shadowing his orher session. This may be done to allow the user to close confidential orsensitive files prior to a help desk technician connecting to make anadjustment to system settings. Rather than displaying theseuser-interface elements as generated by the server, using the techniquesdiscussed herein, the server may intercept and redirect the information,status, permission dialog, or other element to the client, such that theclient's operating system may generate a corresponding user interfaceelement.

Early Authentication (Network-level Authentication) UI

Network-level authentication may comprise security UI prompting forauthentication at the stack level before a remote session is created.This may be done to prevent denial-of-service attacks or brute-forcepassword hacking. Because such authentication takes place prior to thesession being established, server-generated UI elements may not beavailable. Accordingly, it may be preferable to generate theuser-interface on the client prior to the session being initiated, andthen redirecting user credentials to the server once communications areestablished.

Smart Auditor UI

In many embodiments, the client and/or server may provide the ability torecord a session, or record user interaction with a remote session suchas keyboard and mouse input. To provide confirmation and notificationthat recording is underway, it may be desirable to provide a userinterface element, such as a recording or “on-air” light to inform theuser that the session is being recorded. In many embodiments, using thetechniques discussed herein, a server beginning recording may intercepta recording status message on the server and redirect the status to theclient, such that the client may generate a corresponding user interfaceelement in a style consistent with the client-side operating system.

Third Party UIs

Utilizing the techniques discussed herein, any third party UI that canbe decomposed into redirectable elements and is considered to be anaberration from a seamless user experience or is otherwise annoying, mayalso be intercepted, redirected, and presented on the client in auniform and less obtrusive manner. At the same time, theserver-generated UI can be hidden, so it is not visible to the user. Forexample, the Outlook application manufactured by Microsoft, Inc.frequently generates user interface elements to notify a user ofappointments or tasks, request confirmation before deleting meetingrequests, or perform other tasks. These queries, status informationdialogs, or other elements may be intercepted on the server andredirected to a client, such that the client may generate the messagesin a consistent style.

Client-to-Server UI Redirection

Similar to the above-discussed techniques, in many embodiments it may bedesirable to intercept status messages or other UI elements generated bythe client and redirect these elements to the server for generation anddisplay. This may be done where client-side user interface elements maybe inconsistent or confusing in appearance (such as a locally-generatedpopup in one OS style appearing over a full-screen remotely generateddesktop in another style), or confusing in context (such as alow-battery indicator from the local user's laptop appearing over aremote desktop hosted on a non-battery powered server). Several examplesare detailed below.

Live-in Desktop

In many embodiments, a user may view a remote desktop or virtual machinein a full-screen or non-windowed display. This may be referred to as alive-in desktop, indicating that the user's experience “lives” in theremote or virtual desktop rather than their local desktop. Because alive-in desktop covers over local UI elements, such as local taskbars orsystem tray elements, docks, menus, and other elements, if a localapplication or client agent needs to display a user interface element,such user interface element may be hidden behind the live-in desktop.Accordingly, in some embodiments, these user interface elements may beintercepted and redirected to the remote desktop or virtual machinesession, such that the element may be presented using the server's lookand feel. Thus, these elements would then appear in the hosted desktopand be visible to the user.

Referring briefly to FIGS. 4A and 4B, illustrated are block diagramsdepicting embodiments of integration of local and remote applicationwindows. FIG. 4A depicts integration in a full-screen remote desktop,and FIG. 4B depicts integration in a windowed remote desktop. In briefoverview, integrated desktops may include a local application window 400and remote application window 402. In a full-screen mode, the displaymay include a remote desktop environment 406, while in a windowed mode,the display may include a local desktop environment 220. In thefull-screen mode, a remote taskbar 404 generated on the remote host mayinclude buttons representative of local and remote processes, such aslocal application window button 408 and remote application button 410.Similarly, in many embodiments of a windowed remote desktop, the localdesktop environment 220 may include a local taskbar 412, with localapplication window button 408 and remote application button 410. Asdiscussed above, when in full screen mode, the local application windowbutton 408 may be redirected to and generated by the remote host. Thismay be done because the local taskbar 412 may be hidden behind orunderneath the remote desktop environment 406 and/or remote taskbar 404.Similarly, when in windowed mode, the remote application window button410 may be redirected to and generated by the client. This may be donebecause the remote taskbar 404 may be hidden or not transmitted to theclient.

Reverse Seamless

In several embodiments of the technique referred to as Reverse Seamlessapplications, local application windows may be seamlessly integratedwith the full-screen remote desktop, as though they are runningremotely. In some embodiments, the window graphics of thereverse-integrated windows are still generated by the local client. Thisposes the same problems of different look and feel between locally andremotely generated UIs. Accordingly, in one embodiment, by using theredirection techniques discussed herein, user interface elements ofthese client windows such as window titles or menu bars, scroll bars, orother elements may be intercepted and redirected to the server forgeneration. Window contents may still be read from the local logicalvideo buffer (LVB) to display local application output.

For example, in one embodiment, straight reverse-seamless Systrayintegration of some messages might be confusing to the end-user. Asdiscussed above, a “Low battery” message in the Systray of a full-screenvirtual or remote desktop session might be interpreted to mean that theremote virtual machine's battery is dying, even though the virtualmachine does not have a battery. Accordingly, it may be desirable tointercept and redirect such messages for presentation in another format.For example, a modified message or UI element may be displayed toexplicitly indicate that the laptop battery is low.

Improved Localization

In many embodiments, client and server environments may use differentlanguage configurations. For example, the server may be configured toprovide English messages and notifications, while the client operates inSpanish. In one embodiment of the systems and methods discussed herein,the sender may specify resource IDs for message strings in UI elements.When intercepting and redirecting these elements, the resource ID may beprovided, allowing the receiver to use a local resource table toretrieve and present an alternate language version of the message stringas a static translation (e.g., table-based). In many embodiments, thesender may also send the actual text. When the resource ID is unknown tothe receiver or is not supplied, the receiver may utilize a dynamictranslation of the actual text, such as via a local or remote dictionaryor language translation utility. This approach provides for a moreuniform user experience.

In many embodiments, redirection of UI elements may be performed in bothserver-to-client and client-to-server directions, frequentlysimultaneously. Furthermore, different UI elements may requireasynchronous or synchronous operations, and in some cases results mayneed to be returned to the sender. An example may include a dialog boxwith confirmation and cancellation buttons. The dialog box generated atthe sender needs to be intercepted and redirected to the receiver, butmaintained. When the user selects a button on the correspondingreceiver-generated dialog box, the result may be intercepted andredirected back to the sender-generated dialog box, allowing operationsto proceed in accordance with the user's wishes.

Window Mode Switching

In many embodiments, the user may have the ability to switch betweenwindowed operation and full-screen window mode or live-in desktops, asdiscussed above. This may require different redirection of userinterface elements for the best end user experience. For example, if theuser switches the remote connection from windowed to full-screen mode, aredirected UI presented locally at the client may no longer be visible,since it will be hidden behind the full-screen desktop. Accordingly, insome embodiments, the UI may be integrated into the remote session as areverse-seamless window. In other embodiments, the redirection may bepaused or re-redirected to fall-back to server-based rendering.Alternatively, a switch from full-screen to windowed mode could triggerredirection of a UI previously rendered on the server.

Roaming and Reconnect

In many embodiments, remote desktop or machine sessions may includefunctionality for automatic client reconnect (ACR). This may beperformed when there has been a full disconnect from the session, andmay comprise reconnection to the disconnected session. Similarly, inmany embodiments, remote desktop or machine sessions may includefunctionality for roaming, allowing a user to reconnect to adisconnected session either manually or automatically from anotherclient machine. Some redirected UI elements may have been asynchronouslysent to the original client machine, while others might have beensynchronous with a pending user action. In some embodiments, thepreviously sent asynchronous UI elements may not be recreated, such thatthey will not be seen on the new client. This may be done, for example,to prevent a previously redirected low-battery indicator from appearingon a resumed session on a non-battery powered desktop. In manyembodiments, previously sent synchronous/blocking UI elements may berecreated and redirected if no response had been received from theoriginal client prior to the disconnect. For example, if a dialog boxwas presented to the user prior to the disconnect, the dialog box may berecreated and retransmitted to the client for regeneration and display.

Pass-Through and Shadowing Pass-Through

In many embodiments, a user may view multiple nested remote desktop orvirtual machine sessions. For example, a user of a first computingdevice may view a remote desktop of a second computing device, whichitself is displaying a remote desktop of a third computing device. Insome embodiments, the UI elements from the most-upstream client (e.g.,the third computing device), may be redirected to and rendered on theclient at which they are intended to be shown, such as themost-downstream client (e.g., the first computing device). In otherembodiments, such as where the user is viewing a full-screen or live-indesktop, no local UI may be presented. Accordingly, the rendering may beperformed in the remote desktop session, such as the client nextupstream from the user's client. In many embodiments, the exact behaviormay be controlled via policies specified by the user, an administratoror application manufacturer.

Similar to the scenarios discussed above, when a user of a firstcomputing device, such as a help desk technician, shadows a secondcomputing device, this is similar to the first computing deviceconnecting to a remote session of the second computing device. If thesecond computing device is itself viewing a remote desktop or virtualmachine of a third computing device, in some embodiments, UI elementsgenerated by the third computing device may be redirected to the secondcomputing device, rather than passed-through to the first computingdevice. This allows the user to still view these items even when beingshadowed. However, the shadower may not be able to view or interact withthe UI elements.

Accordingly, in many embodiments, the third computing device may beconfigured to redirect UI elements to both the second and the firstcomputing devices, allowing both the shadower and shadowed user to viewand interact with these elements. In other embodiments, redirection maybe disabled such that UI element generation is performed at the thirdcomputing device and presented as application output graphics to thesecond computing device, and by extension, the first computing device.

Referring now to FIG. 3A, illustrated is a block diagram of anembodiment of a system for UI redirection. The following descriptiondiscusses primarily server-to-client UI redirection, but one of skill inthe art may readily appreciate that redirection can also occurclient-to-server, in which case the same concepts apply except that theroles of the client and the server are reversed. In brief overview, aserver or remote computing device 106 may redirect user interfaceelements from a process 206 provided by an operating system manufactureror a third party, an operating system UI element such as a start menu300 or systray or other elements (not shown), or any other element. Insome embodiments, the second agent 204 may call to or read from a UIredirection component 308, which may comprise a library, database, orother logic.

In some embodiments, such as where first process 206 is provided by athird-party manufacturer, the individual elements of UIs generated bythe first process 206 may be redirected using API hooks via a systemlibrary, such as the Windows UI Automation API or the user32.dllprovided as part of the Windows operating system by Microsoft, Inc. SuchAPI calls may include MessageBox, MessageBoxEx, CreateDialog, DialogBox,SetDlgItemInt, SetDlgItemText, or any other API that providesinformation about the format, style, and/or contents of a user interfaceelement. In some embodiments, the hooked APIs may then be redirectedusing UI Redirection Component APIs provided by UI redirection component308. In one embodiment, the native windows in the session generated bythe process or operating system and representing hooked messages boxes,dialogs, or other user interface elements may be hidden, for example viaan API call such as ShowWindow(hWnd, SW_HIDE). This may be done in orderto prevent generation of graphics for them on the remote computingdevice 106 that would have to be transmitted to the local computingdevice 102 for display.

In some embodiments, the UI Redirection Component 308 may make furtheridentifications of the type of session, such as a console session, anICA session, an RDP session, or any other type of session. In otherembodiments, the UI Redirection component 308 may make policy decisions,such as whether redirection is enabled or disabled; whether redirectionis enabled in pass-through mode as discussed above; whether redirectionis enabled during shadowing; or any other policy decisions needed toproperly redirect and generate UI elements. Responsive to thesepolicies, the UI redirection component 308 may direct the operatingsystem of the remote computing device 106 and/or of the local computingdevice 102 to:

-   -   Render the UI locally and natively. In one embodiment, this may        comprise generating an error code to the second agent 204 or a        hooking component of the second agent 204 or UI redirection        component 308, thus triggering fallback to local and native        rendering by the operating system, process, or other user        interface generator.    -   Render the UI locally. In one embodiment, this may comprise        generating UI elements via second agent 204 or a graphics engine        directed by second agent 204, without redirecting UI elements        via the network. This may be done to provide a unified theme for        UI elements generated on the remote computing device.    -   Redirect the UI elements over the Transparent UI Integration        Virtual Channel. In one embodiment, this may comprise        redirecting UI elements via a virtual channel between second        agent 204 and first agent 202 or between remote computing device        106 and local computing device 102 to allow the local computing        device 102 or an operating system of local computing device 102        to render the UI elements using its local look and feel.

In some embodiments, some calls to the UI Redirection Component APIs mayresult in asynchronous/non-blocking calls, such as a call to remote asimple text UI element, while others can be synchronous/blocking, suchas to remote a Yes/No dialog button that requires a response from theuser. In many embodiments, the UI Redirection Component APIs may supportmultiple simultaneous instances of UI redirection (contexts) per remotesession. In a further embodiment, clients of the API may run underdifferent security contexts and both in and out of session space.

The Transparent UI Integration Virtual Channel discussed above may beimplemented in a number of different ways. In some embodiments, thevirtual channel may comprise a generic virtual channel, a dynamicvirtual channel, or a static virtual channel. The choice of virtualchannel may be performed responsive to portability requirements of thesystem. Dynamic virtual channels may provide automatic packetfragmentation/reassembly for large data messages, such as those largerthan 5 KB; automatic multiplexing/demultiplexing of messages formultiple simultaneous instances of UI redirection (contexts) persession. Generic virtual channels may provide advanced features forhosting; providing session notifications; fulfilling securityrequirements; performing common state management between the sides of aconnection; and cache optimization.

In some embodiments, the virtual channel may use a protocol thatprovides flexible UI integration host-to-client. Such a protocol mayinclude asynchronous and synchronous calls with status results. Theprotocol may comprise fields and/or identifiers for different UIelements within a UI container, such as a window. These elements mayinclude text strings, progress bars, buttons, or any other type and formof user interface elements. The protocol may also include class IDs todefine different UI element types, and/or resource IDs to definedifferent instances of UI elements. In other embodiments, the protocolmay also include request IDs to identify unique UI redirectioncontexts/containers and to match requests with responses. In still otherembodiments, the protocol may provide flags or request labels fordifferent actions, such as addition, update, removal of arbitrary UIelements within a context, or any other action. As discussed above, toprovide for language translation and localization, the protocol mayinclude support for different language locales and translation based onResource ID. In many embodiments, the protocol may include support forautomatic packet fragmentation and reassembly for large data andmultiplexing or demultiplexing of different contexts, such asfragmentation flags, sequence or channel IDs, or any other information.

In many embodiments, first agent 202 may include a corresponding UIredirection component 308′ similar to UI redirection component 308. Onthe client side, in many embodiments, the UI redirection component 308′may implement the Transparent UI Integration protocol described above.In one embodiment, UI redirection components 308 may comprise a libraryor DLL for receiving and transmitting virtual channel communications ondifferent contexts or virtual channel instances.

In many embodiments, the Client Virtual Channel Component, first agent202, or UI redirection component 308′ may delegate UI rendering to a UIRendering Platform Abstraction layer provided by shell 214 or firstagent 202. This abstraction layer may have a platform-common interfacebut with different implementations depending on the native operatingsystem. The Platform Abstraction layer may be used to render the UInatively thus ensuring seamless integration with the client's look andfeel.

In a further embodiment, the platform abstraction layer may delegaterendering to an application or process running on the client, or may usea rendering library to render the UI in the client's process space witha consistent look and feel. In still other embodiments, the PlatformAbstraction layer may be used to perform direct translation of text intothe local language using a resource table and the Resource ID of text UIelements. In some embodiments, as discussed above, virtual channeltransmissions may include the actual text of UI elements, which may beused by the client in case the Resource ID is not supplied or isunknown. In yet another embodiment, text can be dynamically translatedusing local or remote dictionaries.

Selective Integration of Visual and Functional Application Features

In many implementations of published applications, there may beusability and security considerations which necessitate selectiveintegration of the applications, as opposed to integration of all localapplications. This selective integration may affect application windows,the system tray, file type association handling, or other UI elements.For example, in many embodiments, only the following windows areexpected to be integrated into the remote desktop or virtual desktopsession (sometimes referred to as a virtual desktop appliance or VDAsession):

-   -   (1) Windows from a published application process that are        launched from that remote or virtual desktop session.    -   (2) Windows from a published application process that are        launched via file type association from the remote or virtual        desktop session.    -   (3) Any windows belonging to a child process of processes from        either #1 or #2.

The same may apply to systray integration or other UI elementintegration of published applications in the remote or virtual desktopsession.

In many embodiments, any other local client windows will remain on theclient desktop and will not be integrated into the desktop of the remoteor virtual desktop session. This may include special windows fromauthentication windows or status messages from windows of the remote orvirtual desktop client application, in some embodiments. However, basedon policy configuration, some windows or system tray items may beallowed to show through the remote desktop, such as an authenticationprompt, user access control elevation prompt, a laptop's battery meter,etc.

In a multi-monitor setup, in many embodiments, separate remote orvirtual desktop sessions can be launched from the same client andassigned to different subsets of monitors (sometimes referred to as amulti-VDA scenario). A client UI may be provided for the user to assignmonitors to the remote or virtual desktop sessions. In some embodiments,a subset of monitors may be preserved for the local desktop. In afurther embodiment, published applications and their child processes maybe associated with their parent remote or virtual desktop sessions fromwhich they were launched. This association may include window andsystray integration, file type association (FTA) handling, and other UIelements. The window association can be managed in different ways:

-   -   Block application transition from VDA-to-VDA, VDA-to-Local or        Local-to-VDA desktops. In other words, the system may prevent        users from dragging windows across VDA session boundaries.    -   Allow full transitions, allowing the user to drag windows across        monitors and VDA sessions.    -   Allow dragging off-screen partially but not fully, as long the        mouse stays in the parent VDA session. Clipping may also be        performed. In one embodiment, as the published application        window is moved, the system may set or limit its window region,        such that it is properly clipped by the VDA session boundary. In        another embodiment, the published application window may be set        to be identified as a child of the VDA session window. In a        further embodiment, the clip-siblings window attribute of the        VDA session window may be set such that the published        application window is automatically clipped by the client        operating system.

In some embodiments, in order to manage window associations properly,the client may maintain a complete hierarchy of parent-childrelationships for processes and their windows. This may bestraightforward for the case when a published application process isdirectly launched by the remote host, or when a published applicationdirectly launches another process via the CreateProcess API or otherinterface. However, it may be more complicated when a publishedapplication is launched via a file type association, in which case thelaunched application may become a child process of the remote shell, orin the case when it is launched via a component object model and becomesa child of a hosted service manager instance. To detect the correctparent-child relationship, in some embodiments, the API used to launchthe published application may be hooked or intercepted to monitorlaunching of applications.

In some embodiments, special handling may be required forsingle-instance applications, such as the Windows Media Player providedby Microsoft, Inc. In one such embodiment, the system may not allowlaunching of an instance of the application on the host, if alreadylaunched as a published application on the client.

In some embodiments, to allow proper working area for publishedapplications for purposes of maximizing/minimizing and restoring windowpositioning, the VDA desktop's work area boundaries may be sent from thehost to the client. Accordingly, published applications may thenmaximize by correctly acknowledging the VDA's taskbar size and positionas opposed to the local desktop's taskbar, which could have a differentposition and/or size.

In many embodiments, local published application windows may be removedfrom the local desktop's taskbar by the client agent or a service of theremote desktop client, and restored upon disconnect/log-off from theremote VDA session. This may be done to help create the false appearancethat published applications are not local, while the VDA is active.

In some embodiments, when the VDA goes into windowed mode, publishedapplication windows may be minimized and removed from the VDA. In oneembodiment, the taskbar entries may stay on the VDA taskbar, while inanother embodiment, they may be removed. When the VDA returns to fullscreen mode, the published applications may be restored to resume normaloperation.

Better-than-Local Experience

In some embodiments, such as where the remote desktop's visual themesare different from the client desktop's visual themes, the methods andsystems discussed herein may provide the opportunity to represent theremote VDA's windows with the local client theme and thus blend themseamlessly with published applications. First, in a server-sideembodiment, client-side settings for the appearance of window titles,borders, font types, sizes, colors, or similar theme settings may betransmitted to the server and used in the session. Second, in aclient-side embodiment, the local client may generate window titles andborders for remote application windows, rather than drawing theseelements from the local video buffer.

This also allows features of newer versions of operating systemsexecuted by the local desktop to be applied to windows of remote desktopapplications running older or different versions of the operatingsystem. For example, Windows 7 by Microsoft, Inc. includes various UIanimations and features including snap-to-side, shake, glass, etc. Bycreating all of the remote desktop's windows as local windows for titlebars and borders as discussed above, these local windows may takeadvantage of these features, even when the remote desktop is executingan older version of Windows, such as Windows XP, or another operatingsystem altogether. Similarly, the same technique can be applied inreverse to allow features of the remote desktop's operating system toapply to local windows. In one such embodiment, a proxy window may becreated on the remote desktop or virtual machine for each window of apublished application or applications on the local client. Graphics ofthese proxy windows, including title bars and borders, may betransmitted to the local client, and merged with the window contentsgenerated by the local application. Accordingly, local publishedapplications running on an older operating system, such as Windows XP,can have the look and feel of a newer operating system, such as Windows7.

Reverse Seamless Application Roaming

In some embodiments, if the remote desktop or virtual machine logs offor shuts down, existing published application windows may also beclosed, via a API call to the local client's window manager, andprocesses associated with these windows may be gracefully exited. Thismay be done to perpetuate the illusion that these windows come from theremote machine.

In other embodiments, on a user-triggered VDA disconnect, the publishedapplication windows may stay open on the client as local windows,effectively orphaning these windows. This may be done to allow the userto continue working off-line. In many embodiments, the user mayreconnect after disconnecting, either from the same or a differentclient. Furthermore, in many embodiments, the published applicationwindows may have been closed prior to reconnection, or becomeunavailable (if reconnecting from a new client). In still otherembodiments, the reconnection may have quickly and silently occurred viaan automatic client reconnection (ACR) system. Accordingly, specificbehaviors for windows may be applied. These behaviors for reintegratingexisting published applications or re-launching them followingreconnection may include:

-   -   (1) No reintegration;    -   (2) Silent and automatic reintegration;    -   (3) Let the user decide by prompting and optionally re-launch        published applications;    -   (4) Gracefully close published applications at previous client        during roaming to a new client; or    -   (5) Re-launch published applications at new client during        roaming, possibly in combination with closing published        applications at previous client.

In many embodiments, user-configurable policies may be used to determinewhich behavior to apply.

Bi-Directional File Type Association (FTA) Integration

In some embodiments, file type associations may be passed from client tohost for published applications, or from host to client or client toclient. In many embodiments, the client may replace all file typeassociations (such as shell-open commands in the registry or directhooking of ShellExecute/ShellExecuteEx and CreateProcess APIs) to pointto a remote desktop client or client agent. The remote desktop client orclient agent may then redirect these commands to the remote desktop orvirtual desktop host, which may perform file type associationarbitration. The hierarchy for arbitration between local and remotecomputing environments may comprise:

-   -   (1) Published file type associations for published applications        (Could be client-native, streamed, ICA, RDP, etc.).    -   (2) Client-enumerated file type associations for published        applications (Could be client-native, streamed, ICA, RDP, etc.).    -   (3) File type associations in the VDA session (Could be        VDA-native, streamed, ICA, RDP, etc)

Generally, in these embodiments, a client application is not engagedeven if it owns the file type associations, unless the application ispublished as a published application. This allows consistent managementby keeping applications associated with the parent VDA and locking downlocal environments. In many embodiments, exceptions are possible basedon policy configurations.

In some embodiments, host-to-client file type association redirectionoccurs when a user clicks on a file in the VDA and it gets redirected toa published application as per (1) or (2) above. In other embodiments,client-to-host file type association redirection occurs when a userinteracts with one published application and launches embedded contentassociated with a VDA-hosted application as per (3) above. For example,in one such embodiment, opening a .doc attachment in a published copy ofthe Outlook email program provided by Microsoft, Inc., may launch aVDA-hosted copy of Microsoft Word.

In some embodiments, client-to-client file type association redirectionmay occur when a user interacts with one published application andlaunches embedded content associated with another published applicationas per (1) or (2) above. For example, opening a .doc attachment in apublished copy of the Outlook email program may launch a published copyof Microsoft Word. Even in such client-to-client file type associationredirection, the redirection is still arbitrated by the VDA. In manyembodiments, this excludes redirection to non-published clientapplications as well as applications published by other non-parent VDAs,providing centralized management. For example, launching of the .docfile will fail if there is no published copy of Microsoft Word orparent-VDA-hosted Microsoft Word, even if there is a local non-publishedcopy of Microsoft Word.

The file type associations discussed above may be communicated betweenthe host and client or agents of the host and/or client via, in someembodiments, a virtual channel of the established remote desktopsession.

Bi-Directional URL Integration

In many embodiments, using similar techniques to those discussed abovewith file type association redirection, uniform resource links (URLs)can also be redirected between the host and client. In theseembodiments, published applications and host VDAs may be located acrossdifferent network environments, including different geographies.Accordingly, users may wish to include URL redirection for severalreasons:

-   -   Access restrictions on web or ftp sites based on locale and/or        user privileges;    -   Different content delivered by web sites based on locale, such        as search engines that deliver different results based on        country of origin of the query;    -   Flash acceleration not working well over high latency links,        such as WANs;    -   Remote Audio/Video Extensions (RAVE) not supporting content in        formats of RealMedia, manufactured by RealNetworks, Inc. or        QuickTime, manufactured by Apple Computers, Inc.; and    -   Lack of Microsoft Silverlight support at a client or host        computer or display via a presentation protocol.

In some embodiments, bi-directional URL redirection may includeapplication-to-web-browser redirection. For example, if a user clicks ona URL link in an e-mail window presented by a published copy ofMicrosoft Outlook, the URL may be redirected to the remote VDA andopened in a browser executing in the remote or virtual desktop. This maybe done using shell-open commands, similar to the file type associationredirection described above.

In other embodiments, URL redirection may include in-web-browserredirection. For example, if a user browses a web page in a web browserexecuting in the remote or virtual desktop and clicks on another linkreferenced from the web page, the link may be redirected to and openedby a published web browser running on the client. In such embodiments,the shell does not get involved. Rather, in these embodiments, thebrowser may handle the links natively. In one embodiment, the URL may beintercepted by a browser helper object or plug-in. Different plug-insmay be required for different browsers. In another embodiment, the URLmay be intercepted by hooking calls to the Windows HTTP Services(WinHTTP) API or a similar API provided by the operating system of theremote or virtual desktop.

In many embodiments, URL redirection may be performed responsive to oneor more policies. These policies may be based on security zones,white-lists and black-lists, or any other information, with therationale of minimizing configurations for the most common requirements.The policies may, in some embodiments, be implemented by URL matching,flexible pattern matching, wild cards, or any other type and form ofpolicy application. For example, in one embodiment of a policy, a whitelist may exist on the client side such that only the listed URLs arehandled by the published web browser from the client. Any other webcontent will be redirected to the browser within the VDA. In anotherembodiment of a policy, a black list may exist on the VDA side such thatthe listed URLs are sent to the published browser and handled from theclient side. Any other web content will be processed within the VDAsession. In some embodiments, URLs in different protocols such as HTTP,MMS, FTP, or any other protocol may be intercepted and redirected.

In some embodiments, in case of redirection errors or errors inaccessing content (e.g. a 404 error received by the redirected-tobrowser), a fallback function may include reassessing the content by theother browser. This may also be helpful where the different browsers mayreceive different error messages, for instance due to being inside afirewall or company network. In one embodiment of client-side fallback,when a URL is in the white list but local client launching fails, thesystem may fallback to VDA launch by redirecting the URL back to theremote VDA browser. In another similar embodiment, when the URL is notin the white list but launch on the VDA browser fails, the system mayfallback to local client launch by redirecting the URL back to thepublished browser on the client. Similarly, in one embodiment of serveror remote-side fallback, when a URL is not in the black list but launchon the VDA browser fails, the system may fallback to local client launchby redirecting the URL back to the published browser. In anotherembodiment, when the URL is in the black list but local client launchfails, the system may fallback to VDA launch by redirecting the URL backto the remote VDA browser.

In some embodiments, web cookies and tokens may also be synchronizedbetween published applications and the virtual or remote desktop. Inaddition, cookies may be sandboxed so they do not affect other browsersessions, including multi-VDA scenarios.

User Authentication Redirection

In many embodiments, it may be desirable to redirect authenticationcredentials from the host to client or vice versa, to allow one systemto execute applications as the user of the other system. For example,the user could be authenticated to the client environment as identity Aor might use an anonymous user account, as in a kiosk type environment.The user may then launch the remote or virtual desktop session andauthenticate as identity B. By default, published applications will runin the context of A. However, in many embodiments, the publishedapplications need to run in the context of B in order to access domainresources similar to applications running in the VDA.

In one embodiment of a solution, the system may use Kerberos over SSPI(Security Support Provider Interface). Following authentication to theVDA, the server agent on the VDA calls SSPI to retrieve a service ticketand encrypted authenticator in an opaque array of bytes, known as SSPIdata. The SSPI data may then be sent to the client agent over a virtualchannel of the remote communications session. The client may call SSPIand pass it the SSPI data obtained from the server agent. SSPI mayauthenticate the user and return a logon token. The client can then usethe logon token to launch published applications in the context of B. Insome embodiments, the client may be in the same domain as the VDA or ina trusted domain of the VDA. In many embodiments, launching publishedapplications under the VDA's user context can be configured via a policyon a per-application basis.

Server Drive and Printer Mapping

Normally, published applications may access local client drives as wellas shared network drives. However, in many instances, these publishedapplications may also need to access the VDA's drives, similar toapplications running in the VDA. Therefore, in one embodiment of thesystem discussed herein, server drives from the parent VDA may beenumerated and accessible in the context of a corresponding clientpublished application. This may be accomplished through enumeration ofthe server drives; mapping of the server drives to network drives orother accessible drives; and redirection of read and write requests tothe mapped drives. In one embodiment, Server Message Block (SMB)protocol may be used to map remote server drives.

Similarly, published applications may have access to local printers andnetwork printers. However, the VDA may have access to printers mapped tothe server, including local and network printers inaccessible by theclient. In one embodiment, VDA printers may be enumerated and mapped tothe client, such that the client may direct print traffic via a virtualchannel of the remote desktop communications. This embodiment maycomprise installing Network Print Provider, Print Service, and UniversalPrint Drivers (UPD), and mapping VDA printers into the client andredirecting print traffic.

In another embodiment, a Universal Print (UP) Client may be installed onthe client machine and a UP Server installed on the VDA. The VDA mayprovide a list of network printers accessible by the VDA in order tofacilitate the network printer discovery at the client. In someembodiments, the UP Server may also be installed on the client. In manyembodiments, the client may use the UP Server to print to networkprinters. Because no local printers are created through thisimplementation, no administrator privileges are required on the clientand no additional printer drivers need be installed.

Additional embodiments of the above discussed methods and systems areincluded in the appendices immediately following this description. Theseembodiments are intended to serve as illustrative examples only. One ofskill in the art may readily envision similar embodiments that do notdepart from the scope of this disclosure.

It should be understood that the systems described above may providemultiple ones of any or each of those components and these componentsmay be provided on either a standalone machine or, in some embodiments,on multiple machines in a distributed system. The systems and methodsdescribed above may be implemented as a method, apparatus or article ofmanufacture using programming and/or engineering techniques to producesoftware, firmware, hardware, or any combination thereof. In addition,the systems and methods described above may be provided as one or morecomputer-readable programs embodied on or in one or more articles ofmanufacture. The term “article of manufacture” as used herein isintended to encompass code or logic accessible from and embedded in oneor more computer-readable devices, firmware, programmable logic, memorydevices (e.g., EEPROMs, ROMs, PROMs, RAMs, SRAMs, etc.), hardware (e.g.,integrated circuit chip, Field Programmable Gate Array (FPGA),Application Specific Integrated Circuit (ASIC), etc.), electronicdevices, a computer readable non-volatile storage unit (e.g., CD-ROM,floppy disk, hard disk drive, etc.). The article of manufacture may beaccessible from a file server providing access to the computer-readableprograms via a network transmission line, wireless transmission media,signals propagating through space, radio waves, infrared signals, etc.The article of manufacture may be a flash memory card or a magnetictape. The article of manufacture includes hardware logic as well assoftware or programmable code embedded in a computer readable mediumthat is executed by a processor. In general, the computer-readableprograms may be implemented in any programming language, such as LISP,PERL, C, C++, C#, PROLOG, or in any byte code language such as JAVA. Thesoftware programs may be stored on or in one or more articles ofmanufacture as object code.

Having described certain embodiments of methods and systems forproviding seamless thumbnails for hosted applications, it will nowbecome apparent to one of skill in the art that other embodimentsincorporating the concepts of the invention may be used.

Section C: Illustrative Embodiment for Seamless Windows Virtual ChannelProtocol

The following presents an illustrative embodiment for seamless windowintegration using a virtual channel protocol. Use of specific limitinglanguage within this Section C should not be imputed to otherembodiments described herein. For example, while certain items “must”occur in this embodiment, those items may be optional or merelyillustrative in other embodiments described herein.

Seamless Windows Interface (also sometimes known as Transparent WindowsInterface or Seamless) is an add-on component for client-side systems,such as the Citrix Receiver for XenApp and XenDesktop manufactured byCitrix Systems, Inc. The seamless virtual channel protocol enhances theclient with windows information, making each remote application appearin a separate resizable window on the client device. Users cannot tellthe difference between local and remote applications; all of them arepresented in the same way. Thus, the user interface is said to be“seamless” between locally and remotely executing applications.

Seamlessness may be especially beneficial in a mixed environment, whenusers need to use local and remote applications during regular businessoperation or remote applications hosted on multiple servers at the sametime.

When the host agent starts, it sends a TWI_PACKET_START packet with someessential host information (that is, version, desktop resolution).

The client responds with a TWI_PACKET_C2H_START_ACK packet, confirmingTWI_PACKET_START and supplying client version/capabilities information.This packet indicates the connection mode the client wants to use,seamless or standard. Based on the host version number and supportedseamless protocol version number, the client determines if it ispossible to enable seamless mode.

If the client wants to enable seamless connection, it sets the Actionfield in the TWI_PACKET_C2H_START_ACK packet to zero and sends aTWI_PACKET_C2H_CLIENTINFO/TWI_PACKET_C2H_CLIENTINFOEX packet with someessential client information.

When the client receives the TWI_PACKET_C2H_START_ACK packet and theAction field is set to zero (indicating that the client wants to switchto seamless mode), the host agent sends a TWI_PACKET_OPEN request toindicate the mode switch and a TWI_PACKET_SYSINFO packet with some extrageneral information (window settings on the host).

When the client receives the TWI_PACKET_OPEN packet, it resets allinternal data structures and enables seamless windows processing.

During a seamless connection, the host agent sends windows information;for example, window position, size, styles, and window text, for all thetop-level windows on the client device. In addition, foreground windowinformation is sent (that is, the foreground window ID).

In accordance with this information, the client creates windows with thesame size/position on the client device. Depending on the connection'sserver type, the window creation flags are set as indicated below:

-   -   In embodiments using an older server type, all seamless windows        are created on the client as frameless and captionless.    -   In embodiments using a newer server type, all seamless windows        are created with the frame and caption styles. However, the        frame and caption sizes are zero, effectively giving the client        full control over the window. The caption style flag may be set        because of Taskbar System menu compatibility.

To support client-side window handling (move/resize), the client mayemulate frame/caption elements by intercepting a WM_NCHITTEST message

In many embodiments, the client tries to perform some operations(windows move/resize) locally, sending update information to the hostlater.

Foreground window changes can happen on both the client and the host, soa set of rules is used to balance foreground window changes.

The Z-order on the client is a superset of the host Z-order (the clientalways has more windows than the host). The host Z-order is reproducedon the client by reproducing the owner/owned relationship among windows,TOP_MOST flag in the window style, and Foreground Window balancing.

In some embodiments, the system allows a user to switch between seamlessand full-screen mode by pressing a hotkey combination, such as Shift+F2,or by using the Connection Center. Note that the full screen mode inthis case is not an original full screen mode, but rather a simulation.If the user connection is opened in a standard full screen mode, allgraphics output goes directly to the screen. If the user connection isoriginally opened in seamless mode and then switched to a simulated fullscreen mode, graphics output goes through a Shadow Bitmap, and is onlythen copied to the screen.

When the user issues a change mode command (the above mentioned hotkeycombination or by using the Connection Center), in some embodiments, theclient sends a TWI_PACKET_C2H_PAUSE packet, asking the host agent tosuspend operation. If the client is already in a simulated full screenmode, it sends a TWI_PACKET_C2H_RESUME packet, asking the host agent toresume operation.

At logoff, the host agent switches itself to a suspended mode and doesnot send information to the client. At reconnection, the host agentsends a TWI_PACKET_START packet again, reporting the host agent state as“already running, reconnect.”

Focus Balancing

Focus change events can occur on both sides of the wire—on the clientand on the host. Therefore, the client and the host agent must use a setof rules to negotiate the focus (foreground window).

For example, in one embodiment with a seamless client and two connectedservers, a user may be working in three remote applications—two on thefirst server and one on the second server—and one local program runningon the client. Five focus transition cases are detailed below:

First, focus goes from the first remote window (first server) to thesecond remote window (first server) because of the mouse click insidethe application window. In this transition, the host agent (firstserver) detects a foreground window change and reports it to the client.The client changes the local desktop, bringing the appropriate window tothe foreground.

Second, the focus goes from the first remote window (first server) tothe second remote window (first server) because of a client-originatedevent; for example, Alt+Tab or a mouse click on a task bar. In thistransition, the client receives an operating system notification in theform of WM_KILLFOCUS/WM_SETFOCUS messages and reports it to the hostagent (first server). The host agent (first server) changes its localdesktop, bringing the appropriate window to the foreground.

Third, focus goes from the first remote window (first server) to thelocal program. In this transition, the client receives an operatingsystem notification in the form of a WM_KILLFOCUS message and reports itto the host agent (first server). Because the local program does nothave a corresponding window on the server, the host agent (first server)sets its own window as a foreground to absorb the focus. To avoidappearance of the host agent window on the client device, this window iscreated outside the visible desktop area.

Fourth, the focus goes from the local program to the first remote window(first server). In this transition, the client receives an operatingsystem notification in the form of a WM_SETFOCUS message and reports itto the host agent (first server). The host agent (first server) changesits local desktop, bringing the appropriate window to the foreground.

Fifth, the focus goes from the first remote window (first server) to thethird remote window (second server). In this transition, there are twoinstances of the seamless client running on the client device, one perserver. Each of these seamless clients is independent of the other anddoes not know about the other (the top-level control over multipleconnections is performed by the Connection Center but it does notinvolve focus balancing). Both clients treat windows created by theother client as local programs; that is, from the first client point ofview, focus goes to the local program; from the second client point ofview, focus goes to the third remote application from a local program.

Echo Suppressing and Simultaneous Focus Change Events

The focus change event can be activated on the host and the clientsimultaneously. For example, the host program will request the focuschange and the user will try to change the focus window at the sametime. In this case, the focus is set in accordance with the lastprocessed request.

On a low-speed connection, there may be significant delay between theuser action and an actual focus change event. In certain circumstances,it might cause focus oscillation or uncontrolled focus oscillation(flipping). The seamless client uses echo suppression to avoid focusoscillation.

After the host agent detects the foreground window change, it alwayscompares the new foreground window with the latest client request. Ifthe focus change event occurs because of the client request, there is noneed to report it to the client and the host agent simply ignores theevent.

The client employs the same technique—it will always compare the newforeground window ID with the latest host agent request and will notreport any host-originated events to the host.

Notification Area Support

On some embodiments of servers, the seamless host agent may create awindow of class type Shell_TrayWnd. Applications communicate with thiswindow to use notification area services. This window handles allnotification messages and forwards these to the client.

If the client operating system is Win9x/NT4 or a newer version, theclient makes use of the local notification area to display, create, orchange host notification area icons in response to messages receivedfrom the host. Any mouse activity over these Systray icons is forwardedby the client to the host.

If the client operating system is NT3.51 or a newer version, the clientcreates uniquely identifiable icons for each host notification area iconcreation or change request from the host. These icons are created asminimized windows so they appear similar to standard NT3.51 iconicwindows.

Process Notifications

The TWI_PACKET_CREATEP and TWI_PACKET_DELETEP packets supportnotification of in-session process creation and termination. Though atfirst this might seem better suited for the Control Virtual Channel, thetight coupling of process and window data make it more efficient to doprocess notification in the Seamless Virtual Channel. In addition,process management functions may also be added in the Control VirtualChannel, and the recipient can reuse the process (and window)information from the Seamless Virtual Channel as needed.

Process Data

The TWI_PACKET_CREATEP packet provides information for sessionprocesses, whether or not they own Seamless windows or icons. A Seamlesswindow or icon can identify their process ownership by means of theProcessID field in the TWI_PACKET_CREATEW_V2 and TWI_PACKET_ICON_V2packets respectively.

Seamless Desktop

When a remote full-screen desktop is in Seamless mode, then the Seamlessvirtual channel may be enabled and every window in the remote desktophas a local Seamless window representation, including the Desktop windowitself and the Taskbar.

The Seamless Desktop mode is enabled at the client when either theTWIMode or RTWIMode ICA Client Configuration settings are set to On fora published desktop.

By itself the Seamless Desktop mode appears to the user just like aregular (non-Seamless) remote desktop, which has a single local windowrepresentation. However, the Seamless Desktop mode serves as a base formultiple features that are built as extensions to it. For example:

-   -   1. Client Hosted Apps and Reverse Seamless: Client Hosted Apps        are separately negotiated over the Control VC. Reverse Seamless        is described in this document.    -   2. Remote Multi-touch support for Desktops: Additional protocol        for Multi-touch support is required, which is not defined yet.        But Seamless Desktop mode will allow for granular Multi-touch        capabilities and input per Window.    -   3. Window Monitoring for Desktops: Allows the ICA Client to use        the Headless (Simulation) API on every window in the remote        desktop.

Reverse Seamless

Client Hosted Apps (CHA) are a type of applications that are run on theremote client machine and are also known as Reverse Seamless (RS) apps.

The rationale behind Reverse Seamless is to achieve 100 percent appcompatibility by allowing “problem” apps to run locally and yet be partof the unified desktop experience. This allows the user to seamlesslyexploit the power of the client, tackle difficult multimedia use cases,device access issues, special locale requirements, etc.

Reverse Seamless ensures that certain apps that run on a client machineappear integrated into a remote full-screen desktop just like regularapps running in the remote desktop itself. The Reverse Seamlessintegration covers many different aspects, both visual and functional,e.g., Start Menu and Desktop Shortcuts, Windows, Alt-Tab, Systray,Taskbar, client-to-host and host-to-client FTA and URL redirection, etc.It means “reversing” many of the existing remote desktop presentationtechnologies.

Reverse Seamless Desktop mode is enabled when the RTWIMode ICA ClientConfiguration setting is set to On. Reverse Seamless Desktop mode is anextension to Seamless Desktop mode, which is described in the previoussection. When the HOST AGENT FLAGS REVERSE SEAMLESS andTWI_CLIENT_FLAG_REVERSE_SEAMLESS flags are negotiated by the client andthe host over the Seamless VC protocol, then the Reverse Seamlesscommands are supported.

In some embodiments, the host negotiated flag may be used to confirmthat the host supports the Reverse Seamless feature. This flag workswith TWI_CLIENT_FLAG_REVERSE_SEAMLESS, which is used by the client totell the host that it supports the Reverse Seamless feature. When bothhost and client support this capability, Reverse Seamless Desktop modeis enabled and the Reverse Seamless commands are supported.

Whenever the host capability flag is set, theHOST_AGENT_FLAGS_STR_UTF8_CAPABLE flag must also be set. In other words,all strings and multi-strings transmitted by Reverse Seamless protocolcommands and corresponding elements are in UTF8.

Similarly, The TWI_CLIENT_FLAG_REVERSE_SEAMLESS flag is used to confirmthat the client supports the Reverse Seamless feature. This flag workswith HOST_AGENT_FLAGS_REVERSE_SEAMLESS, which is used by the host totell the client that it supports the Reverse Seamless feature. When bothhost and client support this capability, Reverse Seamless Desktop modeis enabled and the Reverse Seamless commands are supported.

Whenever this flag is set, the TWI_CLIENT_FLAG_STR_UTF8_CAPABLE flagmust also be set. In other words, all strings and multi-stringstransmitted by Reverse Seamless protocol commands and correspondingelements are in UTF8.

Reverse Seamless is achieved in some embodiments by mixing the nativewindows of CHAs with the Seamless windows from the remote SeamlessDesktop and manipulating the Z-order. Normally the Seamless windowrepresenting the remote Desktop is at the bottom of the Z-order,followed by any combination of native CHA windows and Seamless windowsfrom apps in the remote desktop, including the remote desktop's Taskbar.

The client may run in a multi-monitor scenario where multiple remotedesktop connections might be launched, with each remote desktopoccupying a subset of all monitors. Each remote desktop launched fromthe same endpoint is completely isolated from other remote desktops andthe local desktop itself. In other words, windows and Systray items of aCHA launched from a specific desktop are only integrated in thatdesktop.

In some environments local client applications not published as CHAs mayalso be integrated with one or more remote desktops, e.g., a laptop'sbattery meter. The exact behavior could be application-specific orpolicy-controlled.

Seamless Client Structure and Interfaces

Seamless operations associated with a particular remote session, such aswindow manipulation and focus management, may be implemented inside theThinwire virtual driver or in a separate seamless virtual driver. Alltop-level seamless functions dealing with all remote sessions, such asapplication launch, client desktop settings, monitoring, and seamlessuser interface (UI) (Connection Center), may be implemented as a part ofthe client UI (WFCRUN).

In some embodiments, the seamless client may also use a new enhancedinter-process communication (IPC) mechanism for all internal clientcommunication, including seamless operations. Common top-level seamlessfunctions supported by the Connection Center may use the enhanced IPCmechanism to obtain session information, including seamless-specificinformation and to execute commands including seamless-specificcommands.

The Connection Center may also provide an external IPC for communicationwith other client UI components.

Window Control Messages

In some embodiments, packets may be sent from the host to set a ReverseSeamless window to the foreground. In other embodiments, the host maysend packets to the client to set control information related to aReverse Seamless window, such as to show or hide the window, maximize,minimize or restore the window, or any other similar attributes.

In still other embodiments, packets may be sent from the host inresponse to a request from the client to specify the status of or returncode from processing of a command at the host, such as a system trayinteraction. Responses may indicate success, failure, and may include asuccess or failure code in some embodiments.

In still yet other embodiments, packets may be sent to the client totransmit a system tray message to a reverse seamless or publishedapplication. The system tray message may be a result of a shell actionon a system tray icon in the host environment. These message may includea unique sequence number to identify the message.

In yet still other embodiments, the host may send packets to the clientto configure the client's shell work area. In the case of amulti-monitor environment, multiple work area elements may be supplied,each representing the configuration for a specific monitor.

Similarly, in some embodiments, packets may be sent from the client tocreate a Reverse Seamless (RS) window stub representation in the hostsession, integrate it in the host's taskbar, etc. This command may besent to the host when a Client Hosted App (CHA) creates a new RS window.

In some embodiments, the CHA and RS feature logic at the client uses theSession GUID to distinguish reconnection to the same session vs.connection to a new session. The client may also use the GUID touniquely identify sessions in scenarios where multiple simultaneoussessions are launched from the same client and, for example, followingreconnection to disconnected session, to reintegrate any orphaned CHAsand their RS windows into the respective session.

In other embodiments, packets may be sent from the client to the host tochange the attributes of a reverse seamless window in the host session.In still other embodiments, the client may send a packet to the host todestroy a reverse seamless window's stub representation in the hostsession, remove it from the host's taskbar, etc. Upon disconnect orlogoff from the host session, in some embodiments, the reverse seamlesswindow's stub representation in the host session is automaticallydestroyed without an explicit packet being sent from the client.

In some embodiments, the client may send a packet to the host to notifythe host that a reverse seamless window has been set to the foreground.In other embodiments, the client may send a packet to the host as anotification that a new process associated with a CHA has been createdat the client. The process may or may not have a RS window associatedwith it at the time this packet is sent. Multiple processes might beassociated with the launch of a single CHA, in which case multiplepackets are sent. In some embodiments, the CHA and RS feature logic atthe client uses the Session GUID to distinguish reconnection to the samesession vs. connection to a new session. The client also uses the GUIDto uniquely identify sessions in scenarios where multiple simultaneoussessions are launched from the same client and, for example, followingreconnection to a disconnected session, to reintegrate any orphaned CHAsand their RS windows into the respective session. Reintegration isachieved by means of the same create window and create process messages.

In other embodiments, the client may send a packet to the host to notifythe host that a process associated with a CHA has been deleted orterminated at the client. In some embodiments, prior to sending suchnotification, the client may send a request to destroy window stubsassociated with each reverse seamless window, if any, associated withthe same ProcessID. Upon disconnect or logoff from the session, theCHA's process information in the host session may be automaticallydeleted without a notification from the client, in many embodiments.

In still other embodiments, the client may send packets to the host toset control information related to a reverse seamless window, such as toshow or hide, minimize, maximize or restore a window, etc. In yet stillother embodiments, the client may send a packet to the host as a commandfor a reverse seamless system tray icon. The command may contain shellnotification data, and may be uniquely identified by a client generatedsequence number or ID. The sequence number may be mirrored in anyresponse from the host. This allows for multiple pending requests, andasynchronous or out-of-order responses. Notifications may includeballoon icons, icon bitmaps, status messages, interactions with an iconor message, or other information. In some embodiments, a notificationmessage may be sent when a mouse event or hover occurs in the boundingrectangle of a reverse seamless system tray icon in the host session,when a reverse seamless icon is selected or activated with the keyboard,or when those actions occur in a balloon notification.

Section D: Reverse Seamless Functionality Description

Core Reverse Seamless Functionality

In some embodiments, on clicking a Reverse Seamless Application shortcutin the virtual desktop, remote desktop, or virtual desktop appliance(referred to generally as a VDA) session start menu, the application mayrun from the client device but will display within the VDA session as ifrunning inside the session.

The Reverse Seamless application window can be positioned above, below,or in-between the remote windows on the VDA. In many embodiments, theReverse Seamless application window follows the same rules as the remotewindows on the VDA. In some embodiments, the Reverse Seamlessapplication can also be launched using a command line from the VDAsession.

Shortcuts of the published Reverse Seamless applications may be placedin the user start menu in the VDA session. Icons from the actualapplications may be used for the shortcut icons. If the publishedReverse Seamless applications are not available from the client, theymay be indicated as not available using visual methods, such as grayingout the icon, drawing an X through the icon, or other methods.

Application/VDA Association

In some embodiments, the following windows are integrated into the VDAsession:

-   -   1. Windows from a Reverse Seamless application process that was        launched from that VDA.    -   2. Windows from a Reverse Seamless application process that was        launched via FTA from the VDA.    -   3. Any windows belonging to the child process of the processes        from either #1 or #2.        Any other local windows may remain on the client local desktop.        This includes special windows from user interfaces related to        the client agent or session receiver application, as long as the        local windows are not set to always-on-top.

RS Window Minimize and Maximize Behavior

In some embodiments, minimizing the Reverse Seamless Application windowwill minimize it to the taskbar in the VDA session, if applicable.Clicking on the taskbar entry should restore the window.

Similarly, maximizing the Reverse Seamless Application window willmaximize it in the VDA session, covering the appropriate available workarea. It should be maximized to the correct monitor just like any otherremote windows in the session.

Taskbar Integration

In some embodiments, multiple instances of the Reverse SeamlessApplication will abide by the Taskbar settings of the VDI desktop. Thatis, application grouping, ordering, and other features should functionthe same way for the Reverse Seamless Application as it does for thelocal applications within the VDA Session. However, in many embodiments,specific rules may be applied to grouping of windows, described below.

1. RS-with-RS App Windows. RS-with-VDA Hosted App Windows

In some embodiments, windows from the same RS application process may begrouped in a single Taskbar entry, but even though it is desired, it isnot strictly expected to group RS windows with the remote windows fromthe corresponding app on the VDA. This may include grouping of RS appwindows with pinned shortcuts to VDA Hosted apps on a Microsoft Windows7 VDA Taskbar.

2. Pinned Shortcuts to RS Published Apps

In some embodiments, RS Published Apps may be available in the VDA'sStart Menu and will also be pinnable on a Windows 7 VDA Taskbar. In someembodiments, pinned shortcuts to RS Published Apps may be grouped withthe running instances of RS App Windows, or VDA hosted apps, or pinnedshortcuts to VDA Hosted apps.

3. Pinning of RS App Windows to Win7 VDA Taskbar.

In some embodiments, pinning of RS Application Windows may not beallowed within the VDA taskbar. In these embodiments, the defaultTaskbar Tasks for a RS App Window on a Windows 7 VDA may be limited tojust a “Close window” task. In other embodiments, during grouping withVDA Hosted App Windows, the Destination List of the VDA Hosted app willbe reused and any actions on it will apply only to the VDA app.

URL Redirection

In some embodiments, URLs may be redirected both client-to-VDA andVDA-to-client subject to policy configurations for both. The ReverseSeamless application may redirect web contents to the applicationsinstalled in the VDA session. For example, if a media playerapplication, running as a Reverse Seamless application, displays a URLlink, clicking the link may launch the web browser installed within theVDA Session.

In some embodiments, when the URL gets redirected, the target for theURL launch may honor default browser settings (i.e. which web browser ofa plurality of installed web browsers is to be used by default).

The policies will be implemented by URL matching. The following listswill be created to direct the redirection behaviors:

-   -   1. A white list on the client side: Only the listed URLs are        handled by the Reverse Seamless browser from the client. Any        other web content will be redirected to the browser within the        VDA.    -   2. A black list on the VDA side: The listed URLs are sent to the        Reverse Seamless browser and handled from the client side. Any        other web content will be processed within the VDA session.

The policies may also include flexible pattern matching to allow forwildcards; support for FTP URLs; or other features. In some embodiments,redirection may include URL redirection within a browser or a web-basedapp. In a further embodiment, systems for this internal redirection mayinclude Browser Helper Objects, while in another further embodiment,internal redirection may include lower-level interception of WinHTTPcalls. In some embodiments, upon error, such as a 404 error from a webbrowser, the redirection may include fallback functionality to allow thesource browser to attempt to access the URL. This may succeed because ofdifferent network accessibility, or may result in a potentially moremeaningful error message to the end user as compared to straight denialof service. This fallback may happen as follows:

-   -   a. Client: URL is in white list but local client launch fails.        Fall-back to VDA launch over presentation protocol. URL is not        in white list but launch on VDA over presentation protocol        fails. Fallback to local client launch.    -   b. VDA: URL is not in blacklist but launch on VDA fails.        Fall-back to local client launch over presentation protocol. URL        is in black list but local client launch over presentation        protocol fails. Fall-back to VDA launch.

File Open/Save Dialog Behavior

In some embodiments, Save/Save As and Open dialogs from the ReverseSeamless application may default to the local client's File System toenable access to the local content, or any other default location theadministrator might have configured, e.g., a network share, which isalso accessible by VDA-hosted apps.

Reverse FTA

In some embodiments, Reverse Seamless applications may be accessible viaFile Type Associations inside the VDA session. In these embodiments, oneor more client file type associations are modified to point to the VDA.Arbitration and association then occurs at the VDA. In the VDA, FTApriority for arbitration is as follows:

-   -   1) Reverse Seamless FTA        -   a. FTAs that are explicitly published by the admin as part            of publishing the reverse seamless app.        -   b. FTAs retrieved from the client. Only FTAs associated with            a reverse seamless published app will be retrieved. These            client FTAs are only retrieved and used if (1a) is not            published explicitly by the admin.    -   2) FTAs natively present in the VDA. This could include FTAs        provisioned by the Online Plugin in a double hop scenario or        FTAs for apps that are locally installed in the app.

In some embodiments, FTAs present in the client that are not associatedwith a reverse seamless app will not be transmitted and mapped in theVDA. In many embodiments, PTA-launched Reverse Seamless applicationswill be associated with VDA session from which they are launched.

Cookie and Token Redirection

In some embodiments, cookies, tokens, etc. from the VDA/user profile maybe provided to the Reverse Seamless application. For Web/HTTP Cookies,in some embodiments, they will be redirected from VDA to the clientonly, not in the other direction. In other embodiments, cookies andtokens may be provided client-to-host or bi-directionally.

Client Process Control

In some embodiments, the VDA has the capability to monitor ReverseSeamless application processes. A user or administrator can terminatethese processes in the events of failures or hangs, from the VDA.

In one embodiment, a desktop viewer control application may provide auser interface for controlling and monitoring processes. In otherembodiments, the VDA hosts a user interface, launchable via an icon inthe system tray, a start menu item, or other means. Reverse seamlessapplications are enumerated over a virtual channel and represented inthe VDA-hosted UI for remote management. In some embodiments, stubprocesses in the VDA may be created to represent each reverse seamlessapplication, and these processes may be integrated within the VDA'snative task manager.

System Tray Integration

In some embodiments, if the Reverse Seamless application makes use ofthe system tray, the application icon or other element may be integratedinto the system tray on the VDA. In some embodiments, Reverse Seamlessapplications and potential child processes may be integrated into theVDA system tray. Others may not be integrated, based on a policy. Forexample, in some embodiments, the local client Battery Meter may not bepropagated to the VDA's Systray, to avoid confusion to the user, who maythink the VDA's battery is low. However, such policies may beconfigurable based on user or administrator requirements. Similarly,other UI elements that make use of the system tray, such as tooltips andballoons, may also be integrated as well.

Cut & Paste

In some embodiments, Cut/Paste functions between the Reverse Seamlessapplication and those within the VDA Session may be integrated through ashared or copied clipboard or clipboard virtual channel, responsive tosecurity or configuration policies.

Print Screen

In some embodiments, the system may include functionality for capturingscreen images, such as the Print Screen function provided by variousversions of the Windows operating system provided by Microsoft, Inc.These captured images may include images from the Reverse Seamlessapplication and its content, via the following functions:

1. RS App Windows and VDA Desktop

A Print Screen function performed within the VDA Session may capture theReverse Seamless application window and its contents if it is performedby the end user, e.g., using the local client keyboard. This applies toboth Print Screen and Alt Print Screen (on a specific window).

However, if a VDA hosted app uses an API to scrape the screen contents,it will not be aware of the existence of the RS app window, and will notbe able to retrieve it: It will only retrieve the contents of the VDA'sdesktop and all VDA-hosted apps. This allows additional security forpublished applications.

2. Alt Print Screen on VDA Hosted Windows

The client may use a keyboard hook to redirect an event to therespective host window. Responsive to the redirected event, contents maybe retrievable over the Clipboard virtual channel.

Quick Launch/Desktop integration

In some embodiments, shortcuts to the Reverse Seamless applications inthe start menu can be copied to the desktop on the VDA. In otherembodiments, the shortcut can also be moved to the Taskbar quick launcharea. In still other embodiments, the shortcut may be “pinnable” on theTaskbar.

RS App Publishing

In some embodiments, a user interface may be provided in anadministrator console to centrally create and manage Reverse Seamlessapplications for VDAs. In other embodiments, a command line withoptional parameters to the Reverse Seamless application, and optionallyits working directory may be used to publish the application.Environment variables can be used in the command line and workingdirectory. The environment variables will be evaluated from the clientthat launches the Reverse Seamless application. In many embodiments, thecommand line may support any number of arbitrary parameters. In otherembodiments, command-lets, for example PowerShell command-lets,manufactured by Citrix Systems, Inc., may be used to centrally createand manage Reverse Seamless applications.

In one embodiment, icons and FTAs may be preconfigured at publishingtime. Icons and FTAs may be selected or retrieved from the clientmachine if the Administrator Console is run on the client or has directaccess to it. Otherwise, a default icon and FTAs may be set. In anotherembodiment, RS application icons and FTAs may be retrieved from theclient at runtime and populated to the start menu of the VDA.

Disconnection and Logoff

In some embodiments, on a user-triggered VDA disconnect, the ReverseSeamless windows may remain open on the client as local windows.However, reconnection to the session could happen from the same ordifferent client, and the RS apps might have to be closed in themeantime, or unavailable (if a new client), and the reconnect might havequickly happed over ACR. Accordingly, in some embodiments, the clientand host may not remap reverse seamless windows on reconnection. Inother embodiments, the client and host may remap windows silently. Instill other embodiments, the client and host may present a userinterface to the user to allow them to select which applications shouldbe remapped or relaunched.

Similarly, if the VDA logoffs or shuts down, existing Reverse Seamlesswindows may also be closed and the process exited gracefully.

VDA Windowed Mode

In some embodiments, when the VDA goes into windowed mode, ReverseSeamless application windows may be minimized and removed from the VDA.The Taskbar entries may stay on the VDA Taskbar, and the Taskbar buttonsmay keep the default behaviors. When the VDA goes back into full screenmode, the Reverse Seamless applications may be restored and normaloperations resumed.

In one embodiment, if the user attempts to move a Reverse Seamlesswindow off the full screen VDA session boundary into a differentmonitor, the client will make an attempt to stop the movement. Thisbehavior may be made configurable according to a user-set policy.

Scaling

In some embodiments, if Desktop Viewer is used and scaling is enabled,the maximized Reverse Seamless application windows could be made tomatch the VDA resolution.

Section E: Interface Configuration Details

In some embodiments, the systems and methods described herein providesupport for applications that are installed on the client device runningon the published desktop in a so-called reverse seamless mode, when theuser cannot differentiate an application installed on the desktop thathe is using from the application launched from his own client PC. Suchan application is always running on the ICA client machine, but ICAdisplays its window on the VDA in a seamless fashion includingintegration in the taskbar and Alt+Tab switches. These applications maybe referred to as Reverse Seamless Applications or RS Apps.

In many embodiments, RS apps may be published by the administrator via aconsole UI or software development kit (SDK), similar to desktop hostedapplication publishing. In some embodiments, the console UI and SDK willallow specifying file type associations (FTAs) for RS Apps withoutimporting FTA data from the client device.

In one embodiment, the server may receive a list of RS apps at thesession preparation stage and may create shortcuts to these apps in thestart menu of the remote desktop. In some embodiments, to support RSapps in reconnection to a disconnected session, either from the same ora different client, the server may maintain an index of whatapplications were running before the disconnect, and will attempt tore-launch them on the reconnecting client.

In some embodiments, regular hosted applications may be differentiatedfrom reverse seamless applications via an attribute of the applicationobject. The attribute may identify the application as hosted on adesktop, reverse seamless, hosted on a terminal server, streamed, or anysimilar identifier.

In some embodiments, during Client-to-Host launches the host may performspecial security validation on the published applications and theparameters passed to them, if any. Similarly, in other embodiments, thehost may validate both the RS (client-hosted) published apps and theparameters passed to them, if any, during Host-to-Client launches. Inone embodiment, an administrator may publish a RS (client-hosted) appwith optional parameters. Full validation will be done by the host bydefault: Requests to launch an app that is not published may berejected, responsive to a security policy. In some embodiments, requeststo launch an application with parameters that are outside the patterndefined during publishing will be rejected by default.

In some embodiments, an RS app may be available in the Start Menu of theremote VDA. The command line of the VDA session may also supportscripting of RS applications. In these embodiments, a script mayactivate a RS Launcher Agent and send the request to the client, similarto a short-cut on the start menu.

In some embodiments, parameter validation may be disabled to avoidrestricting the scripts configured by the customer. In a furtherembodiment, disabling the parameter validation may only affectvalidation of the parameters and not of the published app itself.Published apps themselves will thus always be validated before sending alaunch request to the client.

In some embodiments, due to security considerations it may beundesirable to import a set of file type associations from the clientmachines. In such cases, the console UI or SDK may provide a way toassociate a file extension with an RS application in a free form.Accordingly, file type associations may be registered in someembodiments based on default FTAs, while in other embodiments, theassociations may be registered by providing information about theextension from the command line. These two behaviors can be implementedby two distinct parameter sets and are applicable to both desktop hostedand reverse seamless applications.

In some embodiments, RS apps appear in the start menu on the remotedesktop as soon as the user logs in. Similarly, the remote desktopshould have FTA for RS apps written to the registry, so that a contentlaunch would call an RS app. To accomplish these, in some embodiments, aclient agent may retrieve a list of applications and FTAs associatedwith each app from a published application broker when a new session iscreated. This information may be provided to the host during the sessionpreparation stage. In some embodiments, once the broker sends a list ofapplications to an agent of the host or client, these applications maybe stored in an in-memory cache of the host or client agent while thesession is being established.

Section F: Control Channel Virtual Channel Protocol

In some embodiments, communications from a client to host and vice versamay be passed via a control virtual channel of an established remotedesktop session. The virtual channel may be implemented, in someembodiments, as a virtual driver, or by using a user-context agentand/or virtual channel interface via a helper DLL. These communicationsmay include session events, which are commands that notify the host orclient of session state, including lock, disconnect, logoff, or otherstates, or a globally unique session ID provided from the host to theclient. In some embodiments, the host may send session eventsimmediately after capability negotiation or at any time afterwards.

In some embodiments, session event notifications may include bindingrequest messages. These may be sent if the host has capability forprocessing session events. In other embodiments, control communicationsmay include binding response messages. These may be sent if the clienthas capability for processing session events and has received a bindingrequest message. Similarly, in other embodiments, control communicationsmay include a binding commit command, if the host has capability forprocessing session events and the client has transmitted a bindingrequest. Capability for processing session events may thus be negotiatedindependently from other capabilities of the client and host, allowinguse of these features independently.

Similarly, in many embodiments, communications may include notificationsof capability for executing client hosted applications and processingcommands related to client hosted applications. Client hostedapplications (CHA) are applications that are run on the remote clientmachine. These are sometimes referred to as Reverse Seamless (RS)applications.

The rationale behind Reverse Seamless applications is to achieve 100percent application compatibility in remote presentation systems byallowing “problem” applications to run locally and yet be part of theunified desktop experience. This allows the user to seamlessly exploitthe power of the client, tackle difficult multimedia use cases, deviceaccess issues, special locale requirements, etc.

Reverse Seamless ensures that certain apps that run on a client machineappear integrated into a remote full-screen desktop just like regularapps running in the remote desktop itself. The Reverse Seamlessintegration covers many different aspects, both visual and functional,e.g., Start Menu and Desktop Shortcuts, Windows, Alt-Tab, Systray,Taskbar, client-to-host and host-to-client FTA and URL redirection, etc.

Commands sent between the client and host via the control virtualchannel may be used, in some embodiments, to notify the other deviceabout capabilities for executing client hosted applications; enumeratingapplications available at the client environment; validating theavailability of applications available at the client environment;requesting and transmitting enumeration of application attributes suchas icons or file type associations from the client, or otherinformation.

Responses to these requests, in some embodiments, may include one ormore attributes of an application, including an application ID. This maybe sent to allow the client and host to communicate with regard to aplurality of applications asynchronously, with potentially out-of-orderresponses, without ambiguity or confusion. In other embodiments,sequence IDs may be used for the same purpose. In one embodiment, thehost may send positive sequence IDs to denote a host-initiated request,and the client may send negative sequence IDs to denote aclient-initiated request. Sequence IDs may be unique, although in manyembodiments, they may be reused after processing the response to arequest is complete.

In some embodiments, the responses may also include an icon of theapplication. In one embodiment, multiple icons in different sizes may besent from the client to the host, allowing scaling on the host asnecessary. In some embodiments, default icons may be used forapplications.

Commands sent via the virtual command channel may also include, in someembodiments, commands to request the launch of a CHA or a resource suchas a URL. The host may, in these embodiments, send these commandsimmediately after capability negotiation, or at any time afterwards. Thecommands to request the launch of a CHA may comprise a direct launchrequest, which may occur when a user clicks on a CHA shortcut in thehost's start menu, desktop, or other interface element, or when anexplicit launch is triggered on a command line in the host session. Inother embodiments, the commands to request the launch of a CHA maycomprise a file type association launch, such as where a user hasclicked on a document via the virtual or remote desktop environmentsupplied by the host. In still other embodiments, the commands torequest the launch of a CHA may comprise a URL, such as where a user hasclicked on a URL listed in an email or other document viewed on aprogram executed by the host and displayed at the client via apresentation protocol.

File type associations may also be looped back to the client via thecontrol virtual channel, such as when a user interacts with one CHA andlaunches embedded content associated with another CHA. For example,opening a .doc attachment in a CHA copy of Microsoft Outlook may launcha CHA copy of Microsoft Word. The file type association may still bearbitrated by the host. Accordingly, in these embodiments, the clientmay send a request to the host with the file type. The host may respondwith a launch response targeting the associated application. The hostmay further send a launch request, mirroring the launch request from theclient with the file type resolved to the associated application. Theclient may then launch the application and document, and respond to thehost with a status code. In some embodiments, failure of the client torespond with a status code may be interpreted as failure, after theexpiration of a time-out timer.

Client Hosted Apps Request/Response Sequence Numbers

In some embodiments, a ICACC_CHA_APP_LAUNCH_REQUEST_DATA structure maybe shared between host and client app and URL launch requests. TheSequenceNumber field of the ICACC_CHA_APP_LAUNCH_REQUEST_DATA structurespecifies the request sequence number, discussed above. In someembodiments, when the host generates the request, SequenceNumber must bea positive value and when the client generates the request,SequenceNumber must be a negative value. However, during a loopbacklaunch, as discussed above, the host mirrors in its launch request theoriginal client-generated sequence number, which is negative. In manyembodiments, sequence numbers must be unique but could be reused afterprocessing the response to a request is complete. The same sequencenumber is mirrored in the response. In case of multiple requests, thesequence number allows the receiver (client or host) to sendasynchronous and potentially out-of-order responses.

Client Hosted Apps FTA Resolution

In some embodiments, the FTAs for some CHAs are admin-configured. TheFTAs for the rest of the CHAs are client-enumerated. The host owns theprocess of resolving FTAs:

-   -   CHAs have precedence over host apps.    -   For CHA FTAs, if an admin-configured or client-enumerated FTA        applies to more than one CHA, precedence is given to the CHA        whose FTA is admin-configured.    -   If the administrator has mistakenly configured the same FTA for        more than one CHA, then the first CHA in enumeration order that        has the FTA is chosen.    -   The client guarantees that client-enumerated FTAs are unique,        which is what happens by default on a Windows OS client system.        This is true unless FTAs are reassigned in the client        environment during the CHA enumeration process, e.g., if a CHA        is being installed in the meantime. This is a rare race        condition and if it does occur, precedence is given to the first        CHA in enumeration order that has the FTA.

Session GUIDs

In many embodiments, a globally unique identifier may be used toidentify each remote host session. A new GUID may be generated for eachnew remote host session and can be persisted between sessiondisconnects/reconnects. In such embodiments, the CHA feature logic atthe client uses the session GUID to distinguish reconnection to the samesession vs. connection to a new session. The CHA feature also uses theGUID to uniquely identify sessions in scenarios where multiplesimultaneous sessions are launched from the same client and, forexample, following reconnection to disconnected session, to reintegrateany orphaned CHAs into the respective session.

URL Redirection White Lists and Black Lists

URL redirection white list policies may be communicated via the virtualcommand channel from the remote host to the client. In one embodiment,the URL White List is a multi-string of URLs, which specify the URLsthat the client is allowed to launch in its local environment. The listmay contain complete URLs or wild cards. Any URL that does not match theURL White List may be redirected for launching at the host.

Similarly, in these embodiments, the host may have a separate URL BlackList, which specifies URLs that are not allowed to launch at the hostand are redirected for launching at the client. The URL Black List ishost-side only and is not communicated to the client. However, the hostadds from the Black List to the White List any entry that is not alreadyin the White List. Thus the host ensures that the URL White List sent tothe client is always a superset of the URL Black List, i.e., either thesame or bigger. This is done to prevent infinite loops in redirection.

In many embodiments, the URL White and Black lists are the same but theURL White List could also be a proper superset (bigger), in order toinclude URLs that are allowed to be launched at both client and host,i.e., there is no need for redirection. In some embodiments, the clientuses a White List and the host uses a Black List in order to minimizeconfiguration, since most URLs are preferred to launch at the host. Inmany embodiments, black lists and white lists may include wild cards forpattern or string matching in URLs.

Section G: Transparent User Interface Integration Virtual Channel

In some remote desktop systems, the client may repeatedly display itslogon status indicator dialog boxes (also referred to as dialogs) on thehost server's desktop (such as a XenApp host server, manufactured byCitrix Systems, Inc.). These are presented on the client as topmostwindows. While the logon status indicator dialogs don't steal focus theydo stay on top of all windows and need to be dragged out of the way ifthe user wants to continue interacting with the window that has thefocus while the connection establishment is in progress. There are twologon status indicator dialogs that are presented. The first logonstatus indicator dialog is presented by the client. The second logonstatus indicator dialog is presented by the server during theestablishment of the connection. The client logon status indicatordialog is closed when the server dialog is displayed so to the user itlooks as if it is a single logon status indicator dialog.

Accordingly, a more pleasant user experience may be provided whenpresenting the logon status indicator dialog by using a single dialog onthe client to present all logon status indicator messages regardless ifthey originate from the client or server. The new scheme will present adialog that is not a topmost window so that it may be covered withouthaving to drag it out of the way and which is rendered on the client.

In some embodiments, the enhanced logon status indicator experience isdisplayed to the user regardless of whether a client receiver componentis or is not running. When the component is not present, the indicatorstill provides a consistent launch experience that resembles the idealdialog as closely as possible. To accomplish this goal, in someembodiments, the component function that is used to present the logonstatus indicator dialog may be provided in a library so that the clientcalls this function directly when the component is not running so thatthe same code can display the dialog in either circumstance. Thefunction may be provided as part of a common library, or DLL. FIG. 5illustrates a legacy logon status indicator dialog (host-generated andhost-rendered). FIG. 6 illustrates a logon status indicator dialog(host-generated but client-rendered) according to an illustrativeembodiment.

The visual appearance for the logon status indicators is a dialog box,which indicates that the connection is being established. In someembodiments, this dialog box should only be displayed if the connectiontakes over 4 seconds to connect. If the user wants more detail as towhat the current status is in regards to establishing the connectionthey may click on a “More Information” button. The logon statusindicator dialog also contains a “Cancel” button that is available tothe user when they are allowed to cancel the connection. At some pointduring the connection process the user is not allowed to cancel theconnection and must wait for it to complete. The user interfacebehavior, which includes the visual appearance as well as the 4 secondtime out and “More information” functionality, is defined by the logonstatus indicator display mechanism.

Virtual Channel Purpose

In order to perform the work of the wire data transmission needed toimplement the requirements of the Status Indicators Channel, a newVirtual Channel will be created to pass text messages and other UIcomponent metadata (not bitmaps) between client and host. As mentionedpreviously, the virtual channel used will be the Transparent UserInterface Integration virtual channel. The first use of this channelwill be for logon status indicator messages from the host to the client.While any text message may be sent over the new virtual channel, thechanges associated with this specification only pertain to logon statusindicator messages. Any other messages generated on the server outsideof logon status indicator messages will not be sent to the client to berendered locally and will continue to be rendered on the server. Logonstatus indicator messages are specifically those messages that aredisplayed when the connection is being established between the clientand host. Examples of messages that are not considered logon statusindicator messages include license annoyance messages, licensing errormessages, and Windows logon prompts.

Client Availability

For a client to receive logon status indicator messages it must load theVirtual Channel driver. If the client loads the Status Indicator Channelthe server will open the channel and attempt to send all logon statusindicator messages to the client with the expectation that the clientwill display the messages to the user. The server will not display anylogon status indicator messages if it succeeds in opening the VC. If theclient does not load the Status Indicator Channel the server maycontinue to present the logon status indicator messages in anon-integrated method.

Performance

In order to address any potential impact to performance, in someembodiments, the logon status indicator messages may be sent over thenew virtual channel asynchronously. This means that the server will notknow if the client does not receive a message. It also has the potentialthat the connection can become fully established before all of the logonstatus indicator messages have been displayed on the client.

Localization

In many embodiments, the logon status indicator dialogs are rendered onthe client computer using the language that the client has beenconfigured through the OS to display. The protocol allows for a messageID for the caption and title to be sent from the server to the client sothat the client can retrieve the message locally. The protocol alsoallows for the caption and title text to be sent from the host to theclient. This text is used by the client only if the client cannot locatethe message locally using the message ID. If the client displays thecaption and/or title using the text that was sent from the server thistext will be in the local language as defined on the server and may notbe the same language as what is being used on the client machine.

Shadowing

In the case of shadowing by an administrator or other user, or when afirst user can view a second user's remote session, in most cases a userwill not be shadowed until they have completed logon. If shadowing doesoccur before the logon is complete the shadowee will not see any logonstatus indicator messages presented directly on the client. In someembodiments, the prompt asking the user if it is okay for them to beshadowed will not be generated locally and will continue to be displayedthrough the server.

Pass Thru

Pass-thru sessions may refer to instances where a user of a firstcomputing device views a remote session of a second computing device,itself viewing a remote session of a third computing device. Indicatorsmay be passed from the third device through the second device forgeneration and display at the first device. In some embodiments, logonstatus indicators will be displayed in the session making a pass-throughconnection. Side effects include the rendering of a systray icon by thehost session of the pass-through session. In other words, in apass-through session, the logon status indicators rendered on Server1while connecting to Server2 will continue to be rendered on Server1 andremoted to the original client.

Mode Switching

In some embodiments, if a user changes modes between windowed and fullscreen, messages will continue to be displayed in the manner that theywere when the original state was established. So, for example if thesession was in full screen to start with, messages will be displayed bythe server (not through Citrix Receiver). If the user switched towindowed the messages will continue to be displayed by the server. Theopposite is true if the user starts off in windowed mode. The messagesare displayed through Citrix Receiver if Citrix Receiver is running. Ifthe user switched to full screen the messages will continue to bedisplayed through Citrix Receiver locally on the client and thereforewill not be seen by the user.

In some embodiments, the Status Indicator Channel may provide a conduitfor data associated with dialogs and message boxes. This supportspresenting both client and host-generated logon status indicatormessages in a unified fashion directly on the client.

This feature may be used to provide a standardized channel as a conduitfor host status and progress messages. This allows unification of thelook and feel of host messages.

In some embodiments, Status Indicator Channel messaging may be moreefficient and reduce bandwidth requirements for host-clientcommunications by replacing multiple dialog bitmaps with a few stringsand integers.

Transactions

Handshake: The client-to-host (C2H) structure of the virtual channel(VC) may be delivered as part of a basic 3-way handshake.

Virtual Channel (VC) Negotiation: The VC Capability negotiation is goingto use a 3-way handshake. The host-to-client (H2C) request and C2Hresponse result in one host-client synchronous roundtrip. The H2C commitmay be asynchronous.

Message Delivery: In many embodiments, the Status Indicator Channel mayuse the Transparent User Interface Integration VC's asynchronousprotocol mode.

Since the logon status indicator messages are delivered asynchronously,e.g., queued up and delivered on a separate thread, then overall logonspeed will not be affected in any measurable way. The only adverseeffect may be that the first only host status indicator message will bedelivered to the ICA Client with an extra delay of up to 2 host-clientroundtrips. This depends on when the message is generated relative toloading of the host-side status indicator UI renderer module and itsability to initiate VC communication. There should be no delay indisplaying any subsequent status indicator messages. However, apotential delay in the first message could cause it to be short-lived,i.e., immediately clobbered by the next message. Accordingly, in someembodiments, the messages may be delivered synchronously. In this case,logon speed will be adversely affected by 2 host-client roundtrips (Theexact time will depend on the connection latency).

Smart Auditor

In some embodiments, SmartAuditor functionality may allow for therecording and playback of remote desktop sessions. These embodiments mayinclude capturing the updating display of the client, which means thatstatus dialogs delivered via Status Indicator Channel messaging will notbe recorded by SmartAuditor. In one embodiment, to provide recordingcapability, logon status indicator messaging may be disabled when SmartAuditing is enabled. Although the subject matter has been described inlanguage specific to structural features and/or methodological acts, itis to be understood that the subject matter defined in the appendedclaims is not necessarily limited to the specific features or actsdescribed above. Rather, the specific features and acts described aboveare disclosed as example forms of implementing the claims.

What is claimed is:
 1. A method, comprising: receiving, by a clientdevice and from a remote host, data associated with a user interfacecomponent of an application executing within a virtual machine hosted bythe remote host; and rendering, by the client device, within a userinterface associated with the client device, and with a graphicalappearance based on at least one user interface component of the userinterface associated with the client device, a user interface componentcorresponding to the user interface component of the applicationexecuting within the virtual machine hosted by the remote host.
 2. Themethod of claim 1, wherein the data identifies the application executingwithin the virtual machine, and wherein rendering the user interfacecomponent corresponding to the user interface component comprisesrendering an application icon corresponding to the application executingwithin the virtual machine.
 3. The method of claim 1, wherein the datacomprises an image of the user interface component of the applicationexecuting within the virtual machine, and wherein rendering the userinterface component corresponding to the user interface componentcomprises utilizing the image to render a thumbnail preview of the userinterface component of the application executing within the virtualmachine.
 4. The method of claim 1, wherein the data comprises at leastone of an appointment notification generated by the applicationexecuting within the virtual machine or a task notification generated bythe application executing within the virtual machine, and whereinrendering the user interface component corresponding to the userinterface component comprises rendering a user interface componentcorresponding to the at least one of the appointment notificationgenerated by the application executing within the virtual machine or thetask notification generated by the application executing within thevirtual machine.
 5. The method of claim 1, wherein rendering the userinterface component corresponding to the user interface componentcomprises at least one of: rendering the user interface componentcorresponding to the user interface component in a taskbar of the userinterface associated with the client device, rendering the userinterface component corresponding to the user interface component in adock of the user interface associated with the client device, orrendering the user interface component corresponding to the userinterface component on a desktop of the user interface associated withthe client device.
 6. The method of claim 1, wherein rendering the userinterface component corresponding to the user interface componentcomprises rendering the user interface component corresponding to theuser interface component in at least one of a taskbar of the userinterface associated with the client device or a dock of the userinterface associated with the client device, the method comprising:determining, based on an identifier associated with the user interfacecomponent of the application executing within the virtual machine and aplurality of identifiers associated with the at least one of the taskbarof the user interface associated with the client device or the dock ofthe user interface associated with the client device, whether the atleast one of the taskbar of the user interface associated with theclient device or the dock of the user interface associated with theclient device includes a group corresponding to the applicationexecuting within the virtual machine.
 7. The method of claim 6,comprising, responsive to determining that the at least one of thetaskbar of the user interface associated with the client device or thedock of the user interface associated with the client device does notinclude a group corresponding to the application executing within thevirtual machine: rendering, in the at least one of the taskbar of theuser interface associated with the client device or the dock of the userinterface associated with the client device, a group for the applicationexecuting within the virtual machine; and rendering, in the group forthe application executing within the virtual machine, the user interfacecomponent corresponding to the user interface component.
 8. The methodof claim 6, comprising, responsive to determining that the at least oneof the taskbar of the user interface associated with the client deviceor the dock of the user interface associated with the client deviceincludes a group corresponding to the application executing within thevirtual machine, rendering, in the group corresponding to theapplication executing within the virtual machine, the user interfacecomponent corresponding to the user interface component.
 9. The methodof claim 8, wherein the group corresponding to the application executingwithin the virtual machine comprises a thumbnail preview of a userinterface component of an application executing on the client device.10. The method of claim 9, wherein rendering the user interfacecomponent corresponding to the user interface component comprisesrendering, in the group corresponding to the application executingwithin the virtual machine and adjacent to the thumbnail preview, anapplication icon corresponding to the application executing within thevirtual machine.
 11. A system, comprising: at least one processor; and amemory storing instructions that when executed by the at least oneprocessor cause the system to: communicate, to a remotely located clientdevice, data associated with a user interface component of anapplication executing within a virtual machine hosted by the system; andinstruct the remotely located client device to render, within a userinterface associated with the remotely located client device and with agraphical appearance based on at least one user interface component ofthe user interface associated with the remotely located client device, auser interface component corresponding to the user interface componentof the application executing within the virtual machine hosted by thesystem.
 12. The system of claim 11, wherein the data identifies theapplication executing within the virtual machine, and wherein theinstructions, when executed by the at least one processor, cause thesystem to instruct the remotely located client device to render anapplication icon corresponding to the application executing within thevirtual machine.
 13. The system of claim 11, wherein the data comprisesan image of the user interface component of the application executingwithin the virtual machine, and wherein the instructions, when executedby the at least one processor, cause the system to instruct the remotelylocated client device to utilize the image to render a thumbnail previewof the user interface component of the application executing within thevirtual machine.
 14. The system of claim 11, wherein the data comprisesat least one of an appointment notification generated by the applicationexecuting within the virtual machine or a task notification generated bythe application executing within the virtual machine, and wherein theinstructions, when executed by the at least one processor, cause thesystem to instruct the remotely located client device to render a userinterface component corresponding to the at least one of the appointmentnotification generated by the application executing within the virtualmachine or the task notification generated by the application executingwithin the virtual machine.
 15. The system of claim 11, wherein theinstructions, when executed by the at least one processor, cause thesystem to instruct the remotely located client device to at least oneof: render the user interface component corresponding to the userinterface component in a taskbar of the user interface associated withthe remotely located client device, render the user interface componentcorresponding to the user interface component in a dock of the userinterface associated with the remotely located client device, or renderthe user interface component corresponding to the user interfacecomponent on a desktop of the user interface associated with theremotely located client device.
 16. One or more non-transitorycomputer-readable media having instructions stored thereon that whenexecuted by a computing device cause the computing device to: receive,from a remote host, data associated with a user interface component ofan application executing within a virtual machine hosted by the remotehost; and render, within a user interface associated with the computingdevice and with a graphical appearance based on at least one userinterface component of the user interface associated with the computingdevice, a user interface component corresponding to the user interfacecomponent of the application executing within the virtual machine hostedby the remote host.
 17. The one or more non-transitory computer-readablemedia of claim 16, wherein the data identifies the application executingwithin the virtual machine, and wherein the instructions, when executedby the computing device, cause the computing device to render anapplication icon corresponding to the application executing within thevirtual machine.
 18. The one or more non-transitory computer-readablemedia of claim 16, wherein the data comprises an image of the userinterface component of the application executing within the virtualmachine, and wherein the instructions, when executed by the computingdevice, cause the computing device to utilize the image to render athumbnail preview of the user interface component of the applicationexecuting within the virtual machine.
 19. The one or more non-transitorycomputer-readable media of claim 16, wherein the data comprises at leastone of an appointment notification generated by the applicationexecuting within the virtual machine or a task notification generated bythe application executing within the virtual machine, and wherein theinstructions, when executed by the computing device, cause the computingdevice to render a user interface component corresponding to the atleast one of the appointment notification generated by the applicationexecuting within the virtual machine or the task notification generatedby the application executing within the virtual machine.
 20. The one ormore non-transitory computer-readable media of claim 16, wherein theinstructions, when executed by the computing device, cause the computingdevice to at least one of: render the user interface componentcorresponding to the user interface component in a taskbar of the userinterface associated with the computing device, render the userinterface component corresponding to the user interface component in adock of the user interface associated with the computing device, orrender the user interface component corresponding to the user interfacecomponent on a desktop of the user interface associated with thecomputing device.